Accessing geospatial data the easy way (Python)

The access to geospatial data has changed significantly over the past decade. Data has traditionally been accessed by downloading several files to a local computer, then analyzing them with software or programming languages. It has always been difficult to access analysis-ready datasets due to the diversity of data formats (NetCDF, Grib2, Geotiff, Shapefile, etc.) and the variety of access protocols from different providers (Opendap, HTTPS, SFTP, WPS, API Rest, Datamarts, etc.). Beyond that, with the ever-increasing size of geospatial datasets, most modern datasets cannot even fit on a local computer, limiting science’s progress

The datasets presented here are large-scale analysis-ready cloud optimized (ARCO). In order to implement an entry point for a list of datasets, we have followed the methodology developed by the Pangeo community, which combines multiple technologies: - Data Lake (or S3, Azure Data Lake Storage, GCS, etc.) : distributed file-object storage - Zarr (or alternatively TileDB, COGs) : chunked N-dimensionnal array formats - Dask (or alternatively Spark, Ray, Distributed) : distributed computing and lazy loading - Intake Catalogs (or alternatively STAC) : a general interface for loading different data formats, mostly but not limited to spatiotemporal assets

For more information, please refer to the pangeo’s website

It is important to keep in mind that the majority of the datasets in the catalogue have language-agnostic formats, making them accessible through a variety of programming languages (including Python, Julia, Javascript, C, etc.) that implement the specifications for these formats (such as Zarr, netcdfs (kerchunk), geojson, etc.).

from distributed import Client
import intake
import hvplot.xarray
import hvplot.pandas
from dask.distributed import PipInstall
import xoak
import xarray as xr
import numpy as np
import pandas as pd

Dask client

We use a Dask client to ensure all following code compatible with the framework run in parallel

client = Client()
client

Client

Client-0c257f1c-66a7-11ed-89c7-00224851794f

Connection method: Cluster object	Cluster type: distributed.LocalCluster
Dashboard: http://127.0.0.1:8787/status

Cluster Info

LocalCluster

a6549329

Dashboard: http://127.0.0.1:8787/status	Workers: 2
Total threads: 2	Total memory: 6.78 GiB
Status: running	Using processes: True

Scheduler Info

Scheduler

Scheduler-4ea6bf18-fd4d-4cc0-85a1-d5c7308e900c

Comm: tcp://127.0.0.1:41043	Workers: 2
Dashboard: http://127.0.0.1:8787/status	Total threads: 2
Started: Just now	Total memory: 6.78 GiB

Workers

Worker: 0

Comm: tcp://127.0.0.1:45751	Total threads: 1
Dashboard: http://127.0.0.1:39069/status	Memory: 3.39 GiB
Nanny: tcp://127.0.0.1:41407
Local directory: /tmp/dask-worker-space/worker-a5j1z3_5

Worker: 1

Comm: tcp://127.0.0.1:37239	Total threads: 1
Dashboard: http://127.0.0.1:41017/status	Memory: 3.39 GiB
Nanny: tcp://127.0.0.1:44951
Local directory: /tmp/dask-worker-space/worker-21ho7a07

Intake catalogs

Intake is a lightweight package for finding, investigating, loading and disseminating data. A cataloging system is used to organize a collection of datasets and data loaders (drivers) are parameterized such that datasets are opened in the desired format for the end user. In the python context, multi-dimensional xarrays could be opened with xarray’s drivers while polygons (shapefiles, geojson) could be opened with geopandas.

Here is the URL where you can access the catalog:

catalog_url = 'https://raw.githubusercontent.com/hydrocloudservices/catalogs/main/catalogs/main.yaml'
cat=intake.open_catalog(catalog_url)
cat

main:
  args:
    path: https://raw.githubusercontent.com/hydrocloudservices/catalogs/main/catalogs/main.yaml
  description: Master Data Catalog
  driver: intake.catalog.local.YAMLFileCatalog
  metadata: {}

In order to arrange the collection of datasets, the catalogue itself makes references to various sub-catalogs:

[cat[field]
 for field in list(cat._entries.keys())]

[<Intake catalog: hydrology>,
 <Intake catalog: atmosphere>,
 <Intake catalog: geography>,
 <Intake catalog: climate_change>]

Even though our catalogue is constantly expanding, some datasets are already available. The next sections contain several examples of queries as well as analyses of various ones.

The current (flattened) catalogue is described in the table below. A dataset should be used after consulting the status field. If a dataset has a “dev” flag, it signifies that we are actively working on it and do not recommend using it. It is production-ready if it has a “prod” flag. The “prod” label signifies that the dataset has undergone quality review and testing, however users should always double-check on their own because errors are still possible.

pd.set_option('display.max_colwidth', None)

pd.DataFrame([[field ,
               dataset,
               cat[field][dataset].describe()['description'],
               cat[field][dataset].describe()['metadata']['status'][0]]
              for field in list(cat._entries.keys())
              for dataset in cat[field]._entries.keys()],
            columns=['field', 'dataset_name', 'description', 'status']) \
.sort_values('field')

	field	dataset_name	description	status
1	atmosphere	era5_reanalysis_single_levels	ERA5 hourly estimates of variables on single levels chunked for time series analysis	prod
2	atmosphere	era5_reanalysis_single_levels_spatial	ERA5 hourly estimates of variables on single levels chunked for spatial analysis	dev
3	atmosphere	era5_land_reanalysis_spatial	ERA5-Land hourly estimates on single level chunked for spatial analysis	dev
4	atmosphere	era5_reanalysis_pressure_levels	ERA5 hourly estimates of variables on pressure levels	prod
5	atmosphere	daymet_daily_na	Daymet Data Version 4.0	prod
6	atmosphere	ghcnd_world	Global Historical Climatology Network daily (GHCNd)	dev
7	atmosphere	scdna	SCDNA a serially complete precipitation and temperature dataset for North America from 1979 to 2018	prod
8	atmosphere	20_century_reanalysis_single_levels	NOAA-CIRES-DOE Twentieth Century Reanalysis (20CR) on single levels spanning 1836 to 2015 chunked for time series analysis	prod
9	atmosphere	20_century_reanalysis_single_levels_large_area	NOAA-CIRES-DOE Twentieth Century Reanalysis (20CR) on single levels spanning 1836 to 2015 chunked for spatial analysis	prod
10	atmosphere	20_century_reanalysis_pressure_levels	NOAA-CIRES-DOE Twentieth Century Reanalysis (20CR) on pressure levels spanning 1836 to 2015 chunked for time series analysis	prod
11	atmosphere	20_century_reanalysis_pressure_levels_large_area	NOAA-CIRES-DOE Twentieth Century Reanalysis (20CR) on pressure levels spanning 1836 to 2015 chunked for spatial analysis	prod
12	atmosphere	terraclimate	TerraClimate is a dataset of monthly climate and climatic water balance for global terrestrial surfaces from 1958-2019	prod
14	climate_change	rcp45_day_NAM_22i_raw_zarr	NA-Cordex (limited to rcp45 for now... more to come!)	dev
13	geography	melcc_polygons	MELCC basin delimitation	dev
0	hydrology	melcc	CEHQ daily flow and water levels	dev

1) Atmosphere datasets

a) ERA5 single levels

ERA5 is the fifth generation ECMWF atmospheric reanalysis of the global climate covering the period from January 1950 to present. ERA5 is produced by the Copernicus Climate Change Service (C3S) at ECMWF.

Reanalysis combines model data with observations from across the world into a globally complete and consistent dataset using the laws of physics. This principle, called data assimilation, is based on the method used by numerical weather prediction centres, where every so many hours (12 hours at ECMWF) a previous forecast is combined with newly available observations in an optimal way to produce a new best estimate of the state of the atmosphere, called analysis, from which an updated, improved forecast is issued. Reanalysis works in the same way, but at reduced resolution to allow for the provision of a dataset spanning back several decades. Reanalysis does not have the constraint of issuing timely forecasts, so there is more time to collect observations, and when going further back in time, to allow for the ingestion of improved versions of the original observations, which all benefit the quality of the reanalysis product.

Property	Values
Temporal extent:	01/01/1979 – 12/31/2020
Spatial extent:	World : [-180, 180, -90, 90]
Chunks (timeseries’s version):	{‘time’: 14880, ‘longitude’: 15, ‘latitude’: 15}
Chunks (spatial’s version):	{‘time’: 24, ‘longitude’: 1440, ‘latitude’: 721}
Spatial resolution:	0.25 degrees
Spatial reference:	WGS84 (EPSG:4326)
Temporal resolution:	1 hour
Update frequency:	In 2023, we will update it weekly

Data access

ds=cat.atmosphere.era5_reanalysis_single_levels.to_dask()
ds

<xarray.Dataset>
Dimensions:    (latitude: 721, longitude: 1440, time: 368184)
Coordinates:
  * latitude   (latitude) float32 90.0 89.75 89.5 89.25 ... -89.5 -89.75 -90.0
  * longitude  (longitude) float32 -180.0 -179.8 -179.5 ... 179.2 179.5 179.8
  * time       (time) datetime64[ns] 1979-01-01 ... 2020-12-31T23:00:00
Data variables:
    t2m        (time, latitude, longitude) float32 dask.array<chunksize=(14880, 15, 15), meta=np.ndarray>
    tp         (time, latitude, longitude) float32 dask.array<chunksize=(14880, 15, 15), meta=np.ndarray>
Attributes:
    institution:  ECMWF
    source:       Reanalysis
    title:        ERA5 forecasts

xarray.Dataset

• Dimensions:
  • latitude: 721
  • longitude: 1440
  • time: 368184
• Coordinates: (3)
  • latitude
    (latitude)
    float32
    90.0 89.75 89.5 ... -89.75 -90.0

    long_name :

    latitude

    standard_name :

    latitude

    units :

    degrees_north

    array([ 90.  ,  89.75,  89.5 , ..., -89.5 , -89.75, -90.  ], dtype=float32)

  • longitude
    (longitude)
    float32
    -180.0 -179.8 ... 179.5 179.8

    array([-180.  , -179.75, -179.5 , ...,  179.25,  179.5 ,  179.75],
          dtype=float32)

  • time
    (time)
    datetime64[ns]
    1979-01-01 ... 2020-12-31T23:00:00

    standard_name :

    time

    array(['1979-01-01T00:00:00.000000000', '1979-01-01T01:00:00.000000000',
           '1979-01-01T02:00:00.000000000', ..., '2020-12-31T21:00:00.000000000',
           '2020-12-31T22:00:00.000000000', '2020-12-31T23:00:00.000000000'],
          dtype='datetime64[ns]')

• Data variables: (2)
  • t2m
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(14880, 15, 15), meta=np.ndarray>

    long_name :

    2 metre temperature

    nameCDM :

    2_metre_temperature_surface

    nameECMWF :

    2 metre temperature

    product_type :

    analysis

    shortNameECMWF :

    2t

    standard_name :

    air_temperature

    units :

    K

    Array Chunk Bytes 1.39 TiB 12.77 MiB Shape (368184, 721, 1440) (14880, 15, 15) Dask graph 117600 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tp
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(14880, 15, 15), meta=np.ndarray>

    long_name :

    Total precipitation

    nameCDM :

    Total_precipitation_1hour_Accumulation

    nameECMWF :

    Total precipitation

    product_type :

    forecast

    shortNameECMWF :

    tp

    standard_name :

    precipitation_amount

    units :

    m

    Array Chunk Bytes 1.39 TiB 12.77 MiB Shape (368184, 721, 1440) (14880, 15, 15) Dask graph 117600 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (3)
  • latitude
    PandasIndex

    PandasIndex(Float64Index([  90.0,  89.75,   89.5,  89.25,   89.0,  88.75,   88.5,  88.25,
                    88.0,  87.75,
                  ...
                  -87.75,  -88.0, -88.25,  -88.5, -88.75,  -89.0, -89.25,  -89.5,
                  -89.75,  -90.0],
                 dtype='float64', name='latitude', length=721))

  • longitude
    PandasIndex

    PandasIndex(Float64Index([ -180.0, -179.75,  -179.5, -179.25,  -179.0, -178.75,  -178.5,
                  -178.25,  -178.0, -177.75,
                  ...
                    177.5,  177.75,   178.0,  178.25,   178.5,  178.75,   179.0,
                   179.25,   179.5,  179.75],
                 dtype='float64', name='longitude', length=1440))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1979-01-01 00:00:00', '1979-01-01 01:00:00',
                   '1979-01-01 02:00:00', '1979-01-01 03:00:00',
                   '1979-01-01 04:00:00', '1979-01-01 05:00:00',
                   '1979-01-01 06:00:00', '1979-01-01 07:00:00',
                   '1979-01-01 08:00:00', '1979-01-01 09:00:00',
                   ...
                   '2020-12-31 14:00:00', '2020-12-31 15:00:00',
                   '2020-12-31 16:00:00', '2020-12-31 17:00:00',
                   '2020-12-31 18:00:00', '2020-12-31 19:00:00',
                   '2020-12-31 20:00:00', '2020-12-31 21:00:00',
                   '2020-12-31 22:00:00', '2020-12-31 23:00:00'],
                  dtype='datetime64[ns]', name='time', length=368184, freq=None))

• Attributes: (3)

  institution :

  ECMWF

  source :

  Reanalysis

  title :

  ERA5 forecasts

Working with the data

We can quickly choose data subsets in both space and time using xarray. Here, we choose July 19–20, 1996, a period when Quebec saw historically extreme precipitation (Canada). The graphic package hvplot can then be used to track the storm throughout the event.

%%time

da = ds.tp \
.sel(time=slice('1996-07-19','1996-07-20'),
     longitude=slice(-90,-50),
     latitude=slice(60,35))

da \
.where(da>=0.001) \
.load() \
.hvplot(groupby='time',
        widget_type='scrubber',
        widget_location='bottom',
        cmap='gist_ncar',
        tiles='ESRI',
        geo=True,
        clim=(0.001, 0.005),
        width=750,
        height=400)

CPU times: user 4.52 s, sys: 323 ms, total: 4.84 s
Wall time: 10.1 s

Because this zarr’s version of ERA5 is optimised for time series analysis, all historical data can be quickly extracted on a relatively small spatial extent (a point or a polygon for instance) as opposed to working with a collection of netcdf files which is typically extremely compute-intensive for large datasets due to the netcdfs being chunked in the time dimension.

%%time
da = (1000*ds.tp) \
.sel(longitude=-75,
     latitude=45,
     method='nearest')

da.hvplot(grid=True, width=800, height=500, color='blue')

CPU times: user 190 ms, sys: 47.3 ms, total: 238 ms
Wall time: 1.61 s

%%time
da = (1000*ds.tp) \
.sel(longitude=-75,
     latitude=45,
     method='nearest') \
.resample(time='1Y') \
.sum()

da.hvplot.line(grid=True, width=800, height=500, color='blue')* \
da.hvplot.scatter(marker='o').opts(color='black', size=14)

CPU times: user 934 ms, sys: 20.5 ms, total: 954 ms
Wall time: 3.34 s

b) ERA5 pressure levels

ERA5 is the fifth generation ECMWF atmospheric reanalysis of the global climate covering the period from January 1950 to present. ERA5 is produced by the Copernicus Climate Change Service (C3S) at ECMWF.

Reanalysis combines model data with observations from across the world into a globally complete and consistent dataset using the laws of physics. This principle, called data assimilation, is based on the method used by numerical weather prediction centres, where every so many hours (12 hours at ECMWF) a previous forecast is combined with newly available observations in an optimal way to produce a new best estimate of the state of the atmosphere, called analysis, from which an updated, improved forecast is issued. Reanalysis works in the same way, but at reduced resolution to allow for the provision of a dataset spanning back several decades. Reanalysis does not have the constraint of issuing timely forecasts, so there is more time to collect observations, and when going further back in time, to allow for the ingestion of improved versions of the original observations, which all benefit the quality of the reanalysis product.

Property	Values
Temporal extent:	01/01/1979 – 12/31/2019
Spatial extent:	Atlantic Northeast : [-96, -52, 40, 63]
Chunks:	{‘time’: 8760, ‘longitude’: 25, ‘latitude’: 25, ‘level’: 1}
Spatial resolution:	0.25 degrees
Spatial reference:	WGS84 (EPSG:4326)
Temporal resolution:	1 hour
Update frequency:	None

ds=cat.atmosphere.era5_reanalysis_pressure_levels.to_dask()
ds

<xarray.Dataset>
Dimensions:    (latitude: 93, level: 6, longitude: 177, time: 359400)
Coordinates:
  * latitude   (latitude) float32 63.0 62.75 62.5 62.25 ... 40.5 40.25 40.0
  * level      (level) int32 300 400 500 700 850 1000
  * longitude  (longitude) float32 -96.0 -95.75 -95.5 ... -52.5 -52.25 -52.0
  * time       (time) datetime64[ns] 1979-01-01 ... 2019-12-31T23:00:00
Data variables:
    r          (time, level, latitude, longitude) float32 dask.array<chunksize=(8760, 1, 25, 25), meta=np.ndarray>
    t          (time, level, latitude, longitude) float32 dask.array<chunksize=(8760, 1, 25, 25), meta=np.ndarray>
    u          (time, level, latitude, longitude) float32 dask.array<chunksize=(8760, 1, 25, 25), meta=np.ndarray>
    v          (time, level, latitude, longitude) float32 dask.array<chunksize=(8760, 1, 25, 25), meta=np.ndarray>
    z          (time, level, latitude, longitude) float32 dask.array<chunksize=(8760, 1, 25, 25), meta=np.ndarray>
Attributes:
    Conventions:  CF-1.6
    history:      2019-12-18 03:49:32 GMT by grib_to_netcdf-2.14.0: /opt/ecmw...

xarray.Dataset

• Dimensions:
  • latitude: 93
  • level: 6
  • longitude: 177
  • time: 359400
• Coordinates: (4)
  • latitude
    (latitude)
    float32
    63.0 62.75 62.5 ... 40.5 40.25 40.0

    long_name :

    latitude

    units :

    degrees_north

    array([63.  , 62.75, 62.5 , 62.25, 62.  , 61.75, 61.5 , 61.25, 61.  , 60.75,
           60.5 , 60.25, 60.  , 59.75, 59.5 , 59.25, 59.  , 58.75, 58.5 , 58.25,
           58.  , 57.75, 57.5 , 57.25, 57.  , 56.75, 56.5 , 56.25, 56.  , 55.75,
           55.5 , 55.25, 55.  , 54.75, 54.5 , 54.25, 54.  , 53.75, 53.5 , 53.25,
           53.  , 52.75, 52.5 , 52.25, 52.  , 51.75, 51.5 , 51.25, 51.  , 50.75,
           50.5 , 50.25, 50.  , 49.75, 49.5 , 49.25, 49.  , 48.75, 48.5 , 48.25,
           48.  , 47.75, 47.5 , 47.25, 47.  , 46.75, 46.5 , 46.25, 46.  , 45.75,
           45.5 , 45.25, 45.  , 44.75, 44.5 , 44.25, 44.  , 43.75, 43.5 , 43.25,
           43.  , 42.75, 42.5 , 42.25, 42.  , 41.75, 41.5 , 41.25, 41.  , 40.75,
           40.5 , 40.25, 40.  ], dtype=float32)

  • level
    (level)
    int32
    300 400 500 700 850 1000

    long_name :

    pressure_level

    units :

    millibars

    array([ 300,  400,  500,  700,  850, 1000], dtype=int32)

  • longitude
    (longitude)
    float32
    -96.0 -95.75 -95.5 ... -52.25 -52.0

    long_name :

    longitude

    units :

    degrees_east

    array([-96.  , -95.75, -95.5 , -95.25, -95.  , -94.75, -94.5 , -94.25, -94.  ,
           -93.75, -93.5 , -93.25, -93.  , -92.75, -92.5 , -92.25, -92.  , -91.75,
           -91.5 , -91.25, -91.  , -90.75, -90.5 , -90.25, -90.  , -89.75, -89.5 ,
           -89.25, -89.  , -88.75, -88.5 , -88.25, -88.  , -87.75, -87.5 , -87.25,
           -87.  , -86.75, -86.5 , -86.25, -86.  , -85.75, -85.5 , -85.25, -85.  ,
           -84.75, -84.5 , -84.25, -84.  , -83.75, -83.5 , -83.25, -83.  , -82.75,
           -82.5 , -82.25, -82.  , -81.75, -81.5 , -81.25, -81.  , -80.75, -80.5 ,
           -80.25, -80.  , -79.75, -79.5 , -79.25, -79.  , -78.75, -78.5 , -78.25,
           -78.  , -77.75, -77.5 , -77.25, -77.  , -76.75, -76.5 , -76.25, -76.  ,
           -75.75, -75.5 , -75.25, -75.  , -74.75, -74.5 , -74.25, -74.  , -73.75,
           -73.5 , -73.25, -73.  , -72.75, -72.5 , -72.25, -72.  , -71.75, -71.5 ,
           -71.25, -71.  , -70.75, -70.5 , -70.25, -70.  , -69.75, -69.5 , -69.25,
           -69.  , -68.75, -68.5 , -68.25, -68.  , -67.75, -67.5 , -67.25, -67.  ,
           -66.75, -66.5 , -66.25, -66.  , -65.75, -65.5 , -65.25, -65.  , -64.75,
           -64.5 , -64.25, -64.  , -63.75, -63.5 , -63.25, -63.  , -62.75, -62.5 ,
           -62.25, -62.  , -61.75, -61.5 , -61.25, -61.  , -60.75, -60.5 , -60.25,
           -60.  , -59.75, -59.5 , -59.25, -59.  , -58.75, -58.5 , -58.25, -58.  ,
           -57.75, -57.5 , -57.25, -57.  , -56.75, -56.5 , -56.25, -56.  , -55.75,
           -55.5 , -55.25, -55.  , -54.75, -54.5 , -54.25, -54.  , -53.75, -53.5 ,
           -53.25, -53.  , -52.75, -52.5 , -52.25, -52.  ], dtype=float32)

  • time
    (time)
    datetime64[ns]
    1979-01-01 ... 2019-12-31T23:00:00

    long_name :

    time

    array(['1979-01-01T00:00:00.000000000', '1979-01-01T01:00:00.000000000',
           '1979-01-01T02:00:00.000000000', ..., '2019-12-31T21:00:00.000000000',
           '2019-12-31T22:00:00.000000000', '2019-12-31T23:00:00.000000000'],
          dtype='datetime64[ns]')

• Data variables: (5)
  • r
    (time, level, latitude, longitude)
    float32
    dask.array<chunksize=(8760, 1, 25, 25), meta=np.ndarray>

    long_name :

    Relative humidity

    standard_name :

    relative_humidity

    units :

    %

    Array Chunk Bytes 132.23 GiB 20.89 MiB Shape (359400, 6, 93, 177) (8760, 1, 25, 25) Dask graph 8064 chunks in 2 graph layers Data type float32 numpy.ndarray

  • t
    (time, level, latitude, longitude)
    float32
    dask.array<chunksize=(8760, 1, 25, 25), meta=np.ndarray>

    long_name :

    Temperature

    standard_name :

    air_temperature

    units :

    K

    Array Chunk Bytes 132.23 GiB 20.89 MiB Shape (359400, 6, 93, 177) (8760, 1, 25, 25) Dask graph 8064 chunks in 2 graph layers Data type float32 numpy.ndarray

  • u
    (time, level, latitude, longitude)
    float32
    dask.array<chunksize=(8760, 1, 25, 25), meta=np.ndarray>

    long_name :

    U component of wind

    standard_name :

    eastward_wind

    units :

    m s**-1

    Array Chunk Bytes 132.23 GiB 20.89 MiB Shape (359400, 6, 93, 177) (8760, 1, 25, 25) Dask graph 8064 chunks in 2 graph layers Data type float32 numpy.ndarray

  • v
    (time, level, latitude, longitude)
    float32
    dask.array<chunksize=(8760, 1, 25, 25), meta=np.ndarray>

    long_name :

    V component of wind

    standard_name :

    northward_wind

    units :

    m s**-1

    Array Chunk Bytes 132.23 GiB 20.89 MiB Shape (359400, 6, 93, 177) (8760, 1, 25, 25) Dask graph 8064 chunks in 2 graph layers Data type float32 numpy.ndarray

  • z
    (time, level, latitude, longitude)
    float32
    dask.array<chunksize=(8760, 1, 25, 25), meta=np.ndarray>

    long_name :

    Geopotential

    standard_name :

    geopotential

    units :

    m**2 s**-2

    Array Chunk Bytes 132.23 GiB 20.89 MiB Shape (359400, 6, 93, 177) (8760, 1, 25, 25) Dask graph 8064 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (4)
  • latitude
    PandasIndex

    PandasIndex(Float64Index([ 63.0, 62.75,  62.5, 62.25,  62.0, 61.75,  61.5, 61.25,  61.0,
                  60.75,  60.5, 60.25,  60.0, 59.75,  59.5, 59.25,  59.0, 58.75,
                   58.5, 58.25,  58.0, 57.75,  57.5, 57.25,  57.0, 56.75,  56.5,
                  56.25,  56.0, 55.75,  55.5, 55.25,  55.0, 54.75,  54.5, 54.25,
                   54.0, 53.75,  53.5, 53.25,  53.0, 52.75,  52.5, 52.25,  52.0,
                  51.75,  51.5, 51.25,  51.0, 50.75,  50.5, 50.25,  50.0, 49.75,
                   49.5, 49.25,  49.0, 48.75,  48.5, 48.25,  48.0, 47.75,  47.5,
                  47.25,  47.0, 46.75,  46.5, 46.25,  46.0, 45.75,  45.5, 45.25,
                   45.0, 44.75,  44.5, 44.25,  44.0, 43.75,  43.5, 43.25,  43.0,
                  42.75,  42.5, 42.25,  42.0, 41.75,  41.5, 41.25,  41.0, 40.75,
                   40.5, 40.25,  40.0],
                 dtype='float64', name='latitude'))

  • level
    PandasIndex

    PandasIndex(Int64Index([300, 400, 500, 700, 850, 1000], dtype='int64', name='level'))

  • longitude
    PandasIndex

    PandasIndex(Float64Index([ -96.0, -95.75,  -95.5, -95.25,  -95.0, -94.75,  -94.5, -94.25,
                   -94.0, -93.75,
                  ...
                  -54.25,  -54.0, -53.75,  -53.5, -53.25,  -53.0, -52.75,  -52.5,
                  -52.25,  -52.0],
                 dtype='float64', name='longitude', length=177))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1979-01-01 00:00:00', '1979-01-01 01:00:00',
                   '1979-01-01 02:00:00', '1979-01-01 03:00:00',
                   '1979-01-01 04:00:00', '1979-01-01 05:00:00',
                   '1979-01-01 06:00:00', '1979-01-01 07:00:00',
                   '1979-01-01 08:00:00', '1979-01-01 09:00:00',
                   ...
                   '2019-12-31 14:00:00', '2019-12-31 15:00:00',
                   '2019-12-31 16:00:00', '2019-12-31 17:00:00',
                   '2019-12-31 18:00:00', '2019-12-31 19:00:00',
                   '2019-12-31 20:00:00', '2019-12-31 21:00:00',
                   '2019-12-31 22:00:00', '2019-12-31 23:00:00'],
                  dtype='datetime64[ns]', name='time', length=359400, freq=None))

• Attributes: (2)

  Conventions :

  CF-1.6

  history :

  2019-12-18 03:49:32 GMT by grib_to_netcdf-2.14.0: /opt/ecmwf/eccodes/bin/grib_to_netcdf -o /cache/data9/adaptor.mars.internal-1576639468.3861034-13687-32-f628cb0a-a2d6-4032-bbbd-0863a804b958.nc /cache/tmp/f628cb0a-a2d6-4032-bbbd-0863a804b958-adaptor.mars.internal-1576639468.386756-13687-13-tmp.grib

Working with the data

%%time
ds.z \
.sel(longitude=-75, latitude=45, level=[500, 700, 850, 1000]).hvplot(grid=True, by='level')

CPU times: user 1.31 s, sys: 140 ms, total: 1.45 s
Wall time: 15.8 s

c) ERA5-Land

ERA5-Land is the fifth generation ECMWF atmospheric reanalysis of the global climate covering the period from January 1950 to present. ERA5-Land is produced by the Copernicus Climate Change Service (C3S) at ECMWF.

Reanalysis combines model data with observations from across the world into a globally complete and consistent dataset using the laws of physics. This principle, called data assimilation, is based on the method used by numerical weather prediction centres, where every so many hours (12 hours at ECMWF) a previous forecast is combined with newly available observations in an optimal way to produce a new best estimate of the state of the atmosphere, called analysis, from which an updated, improved forecast is issued. Reanalysis works in the same way, but at reduced resolution to allow for the provision of a dataset spanning back several decades. Reanalysis does not have the constraint of issuing timely forecasts, so there is more time to collect observations, and when going further back in time, to allow for the ingestion of improved versions of the original observations, which all benefit the quality of the reanalysis product.

Property	Values
Temporal extent:	01/01/1950 – present
Spatial extent:	North America : [-167, -50, 15, 85]
Chunks (timeseries’s version):	{‘time’: 8760, ‘longitude’: 7, ‘latitude’: 7}
Chunks (spatial’s version):	{‘time’: 24, ‘longitude’: 1171, ‘latitude’: 701}
Spatial resolution:	0.1 degrees
Spatial reference:	WGS84 (EPSG:4326)
Temporal resolution:	1 hour
Update frequency:	In 2023, we will update it monthly

Available in December 2022. Please refer to previous ERA5 examples once the dataset is added to the catalog.

d) Daymet

The Daymet dataset contains daily minimum temperature, maximum temperature, precipitation, shortwave radiation, vapor pressure, snow water equivalent, and day length at 1km resolution for North America. Annual and monthly summaries are also available. The dataset covers the period from January 1, 1980 to December 31, 2020.

Daymet is accessible on Azure in Zarr format; this notebook shows how to access the data using the Planetary Computer’s resources so that it can be read into a xarray dataset.

Property	Values
Temporal extent:	01/01/1980 – 12/31/2020
Spatial extent:	North America
Chunks (timeseries’s version):	{‘time’: 365, ‘longitude’: 584, ‘latitude’: 284}
Spatial resolution:	1 km
Spatial reference:	Custom (‘+ellps=WGS84 +proj=lcc +lon_0=-100 +lat_0=42.5 +x_0=0.0 +y_0=0.0 +lat_1=25 +lat_2=60 +no_defs’)
Temporal resolution:	1 day
Update frequency:	None

Data access

ds=cat.atmosphere.daymet_daily_na.to_dask()
ds

<xarray.Dataset>
Dimensions:                  (time: 14965, y: 8075, x: 7814, nv: 2)
Coordinates:
    lat                      (y, x) float32 dask.array<chunksize=(284, 584), meta=np.ndarray>
    lon                      (y, x) float32 dask.array<chunksize=(284, 584), meta=np.ndarray>
  * time                     (time) datetime64[ns] 1980-01-01T12:00:00 ... 20...
  * x                        (x) float32 -4.56e+06 -4.559e+06 ... 3.253e+06
  * y                        (y) float32 4.984e+06 4.983e+06 ... -3.09e+06
Dimensions without coordinates: nv
Data variables:
    dayl                     (time, y, x) float32 dask.array<chunksize=(365, 284, 584), meta=np.ndarray>
    lambert_conformal_conic  int16 ...
    prcp                     (time, y, x) float32 dask.array<chunksize=(365, 284, 584), meta=np.ndarray>
    srad                     (time, y, x) float32 dask.array<chunksize=(365, 284, 584), meta=np.ndarray>
    swe                      (time, y, x) float32 dask.array<chunksize=(365, 284, 584), meta=np.ndarray>
    time_bnds                (time, nv) datetime64[ns] dask.array<chunksize=(365, 2), meta=np.ndarray>
    tmax                     (time, y, x) float32 dask.array<chunksize=(365, 284, 584), meta=np.ndarray>
    tmin                     (time, y, x) float32 dask.array<chunksize=(365, 284, 584), meta=np.ndarray>
    vp                       (time, y, x) float32 dask.array<chunksize=(365, 284, 584), meta=np.ndarray>
    yearday                  (time) int16 dask.array<chunksize=(365,), meta=np.ndarray>
Attributes:
    Conventions:       CF-1.6
    Version_data:      Daymet Data Version 4.0
    Version_software:  Daymet Software Version 4.0
    citation:          Please see http://daymet.ornl.gov/ for current Daymet ...
    references:        Please see http://daymet.ornl.gov/ for current informa...
    source:            Daymet Software Version 4.0
    start_year:        1980

xarray.Dataset

• Dimensions:
  • time: 14965
  • y: 8075
  • x: 7814
  • nv: 2
• Coordinates: (5)
  • lat
    (y, x)
    float32
    dask.array<chunksize=(284, 584), meta=np.ndarray>

    long_name :

    latitude coordinate

    standard_name :

    latitude

    units :

    degrees_north

    Array Chunk Bytes 240.70 MiB 647.88 kiB Shape (8075, 7814) (284, 584) Dask graph 406 chunks in 2 graph layers Data type float32 numpy.ndarray

  • lon
    (y, x)
    float32
    dask.array<chunksize=(284, 584), meta=np.ndarray>

    long_name :

    longitude coordinate

    standard_name :

    longitude

    units :

    degrees_east

    Array Chunk Bytes 240.70 MiB 647.88 kiB Shape (8075, 7814) (284, 584) Dask graph 406 chunks in 2 graph layers Data type float32 numpy.ndarray

  • time
    (time)
    datetime64[ns]
    1980-01-01T12:00:00 ... 2020-12-...

    bounds :

    time_bnds

    long_name :

    24-hour day based on local time

    standard_name :

    time

    array(['1980-01-01T12:00:00.000000000', '1980-01-02T12:00:00.000000000',
           '1980-01-03T12:00:00.000000000', ..., '2020-12-28T12:00:00.000000000',
           '2020-12-29T12:00:00.000000000', '2020-12-30T12:00:00.000000000'],
          dtype='datetime64[ns]')

  • x
    (x)
    float32
    -4.56e+06 -4.559e+06 ... 3.253e+06

    long_name :

    x coordinate of projection

    standard_name :

    projection_x_coordinate

    units :

    m

    array([-4560250., -4559250., -4558250., ...,  3250750.,  3251750.,  3252750.],
          dtype=float32)

  • y
    (y)
    float32
    4.984e+06 4.983e+06 ... -3.09e+06

    long_name :

    y coordinate of projection

    standard_name :

    projection_y_coordinate

    units :

    m

    array([ 4984000.,  4983000.,  4982000., ..., -3088000., -3089000., -3090000.],
          dtype=float32)

• Data variables: (10)
  • dayl
    (time, y, x)
    float32
    dask.array<chunksize=(365, 284, 584), meta=np.ndarray>

    cell_methods :

    area: mean

    grid_mapping :

    lambert_conformal_conic

    long_name :

    daylength

    units :

    s

    Array Chunk Bytes 3.44 TiB 230.93 MiB Shape (14965, 8075, 7814) (365, 284, 584) Dask graph 16646 chunks in 2 graph layers Data type float32 numpy.ndarray

  • lambert_conformal_conic
    ()
    int16
    ...

    false_easting :

    0.0

    false_northing :

    0.0

    grid_mapping_name :

    lambert_conformal_conic

    inverse_flattening :

    298.257223563

    latitude_of_projection_origin :

    42.5

    longitude_of_central_meridian :

    -100.0

    semi_major_axis :

    6378137.0

    standard_parallel :

    [25.0, 60.0]

    [1 values with dtype=int16]

  • prcp
    (time, y, x)
    float32
    dask.array<chunksize=(365, 284, 584), meta=np.ndarray>

    cell_methods :

    area: mean time: sum

    grid_mapping :

    lambert_conformal_conic

    long_name :

    daily total precipitation

    units :

    mm/day

    Array Chunk Bytes 3.44 TiB 230.93 MiB Shape (14965, 8075, 7814) (365, 284, 584) Dask graph 16646 chunks in 2 graph layers Data type float32 numpy.ndarray

  • srad
    (time, y, x)
    float32
    dask.array<chunksize=(365, 284, 584), meta=np.ndarray>

    cell_methods :

    area: mean time: mean

    grid_mapping :

    lambert_conformal_conic

    long_name :

    daylight average incident shortwave radiation

    units :

    W/m2

    Array Chunk Bytes 3.44 TiB 230.93 MiB Shape (14965, 8075, 7814) (365, 284, 584) Dask graph 16646 chunks in 2 graph layers Data type float32 numpy.ndarray

  • swe
    (time, y, x)
    float32
    dask.array<chunksize=(365, 284, 584), meta=np.ndarray>

    cell_methods :

    area: mean time: mean

    grid_mapping :

    lambert_conformal_conic

    long_name :

    snow water equivalent

    units :

    kg/m2

    Array Chunk Bytes 3.44 TiB 230.93 MiB Shape (14965, 8075, 7814) (365, 284, 584) Dask graph 16646 chunks in 2 graph layers Data type float32 numpy.ndarray

  • time_bnds
    (time, nv)
    datetime64[ns]
    dask.array<chunksize=(365, 2), meta=np.ndarray>

    Array Chunk Bytes 233.83 kiB 5.70 kiB Shape (14965, 2) (365, 2) Dask graph 41 chunks in 2 graph layers Data type datetime64[ns] numpy.ndarray

  • tmax
    (time, y, x)
    float32
    dask.array<chunksize=(365, 284, 584), meta=np.ndarray>

    cell_methods :

    area: mean time: maximum

    grid_mapping :

    lambert_conformal_conic

    long_name :

    daily maximum temperature

    units :

    degrees C

    Array Chunk Bytes 3.44 TiB 230.93 MiB Shape (14965, 8075, 7814) (365, 284, 584) Dask graph 16646 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tmin
    (time, y, x)
    float32
    dask.array<chunksize=(365, 284, 584), meta=np.ndarray>

    cell_methods :

    area: mean time: minimum

    grid_mapping :

    lambert_conformal_conic

    long_name :

    daily minimum temperature

    units :

    degrees C

    Array Chunk Bytes 3.44 TiB 230.93 MiB Shape (14965, 8075, 7814) (365, 284, 584) Dask graph 16646 chunks in 2 graph layers Data type float32 numpy.ndarray

  • vp
    (time, y, x)
    float32
    dask.array<chunksize=(365, 284, 584), meta=np.ndarray>

    cell_methods :

    area: mean time: mean

    grid_mapping :

    lambert_conformal_conic

    long_name :

    daily average vapor pressure

    units :

    Pa

    Array Chunk Bytes 3.44 TiB 230.93 MiB Shape (14965, 8075, 7814) (365, 284, 584) Dask graph 16646 chunks in 2 graph layers Data type float32 numpy.ndarray

  • yearday
    (time)
    int16
    dask.array<chunksize=(365,), meta=np.ndarray>

    long_name :

    day of year (DOY) starting with day 1 on Januaray 1st

    Array Chunk Bytes 29.23 kiB 730 B Shape (14965,) (365,) Dask graph 41 chunks in 2 graph layers Data type int16 numpy.ndarray

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1980-01-01 12:00:00', '1980-01-02 12:00:00',
                   '1980-01-03 12:00:00', '1980-01-04 12:00:00',
                   '1980-01-05 12:00:00', '1980-01-06 12:00:00',
                   '1980-01-07 12:00:00', '1980-01-08 12:00:00',
                   '1980-01-09 12:00:00', '1980-01-10 12:00:00',
                   ...
                   '2020-12-21 12:00:00', '2020-12-22 12:00:00',
                   '2020-12-23 12:00:00', '2020-12-24 12:00:00',
                   '2020-12-25 12:00:00', '2020-12-26 12:00:00',
                   '2020-12-27 12:00:00', '2020-12-28 12:00:00',
                   '2020-12-29 12:00:00', '2020-12-30 12:00:00'],
                  dtype='datetime64[ns]', name='time', length=14965, freq=None))

  • x
    PandasIndex

    PandasIndex(Float64Index([-4560250.0, -4559250.0, -4558250.0, -4557250.0, -4556250.0,
                  -4555250.0, -4554250.0, -4553250.0, -4552250.0, -4551250.0,
                  ...
                   3243750.0,  3244750.0,  3245750.0,  3246750.0,  3247750.0,
                   3248750.0,  3249750.0,  3250750.0,  3251750.0,  3252750.0],
                 dtype='float64', name='x', length=7814))

  • y
    PandasIndex

    PandasIndex(Float64Index([ 4984000.0,  4983000.0,  4982000.0,  4981000.0,  4980000.0,
                   4979000.0,  4978000.0,  4977000.0,  4976000.0,  4975000.0,
                  ...
                  -3081000.0, -3082000.0, -3083000.0, -3084000.0, -3085000.0,
                  -3086000.0, -3087000.0, -3088000.0, -3089000.0, -3090000.0],
                 dtype='float64', name='y', length=8075))

• Attributes: (7)

  Conventions :

  CF-1.6

  Version_data :

  Daymet Data Version 4.0

  Version_software :

  Daymet Software Version 4.0

  citation :

  Please see http://daymet.ornl.gov/ for current Daymet data citation information

  references :

  Please see http://daymet.ornl.gov/ for current information on Daymet references

  source :

  Daymet Software Version 4.0

  start_year :

  1980

Working with the data

Because Daymet has a custom projection, we use xoak library to query some data. It is also possible to regrid or reproject the data to facilitate analysis.

%%time
ds = ds.sel(time=slice('2000-01-01','2001-01-01'))
points = xr.Dataset(
    {
        "lat": 45,
        "lon": -75,
    }
)

da_tmax = ds.tmax
da_tmax.xoak.set_index(["lat", "lon"], "sklearn_geo_balltree")

da_tmin = ds.tmin
da_tmin.xoak.set_index(["lat", "lon"], "sklearn_geo_balltree")

prcp = ds.prcp
prcp.xoak.set_index(["lat", "lon"], "sklearn_geo_balltree")

swe = ds.swe
swe.xoak.set_index(["lat", "lon"], "sklearn_geo_balltree")

(da_tmax.xoak.sel(lat=points.lat,
                  lon=points.lon).hvplot(grid=True,
                                         value_label='daily temperature (degrees C)')* \
da_tmin.xoak.sel(lat=points.lat,
                 lon=points.lon).hvplot(grid=True) + \
prcp.xoak.sel(lat=points.lat,
              lon=points.lon).hvplot(grid=True) + \
swe.xoak.sel(lat=points.lat,
             lon=points.lon).hvplot(grid=True)
).cols(1)

CPU times: user 20.3 s, sys: 1.19 s, total: 21.5 s
Wall time: 4min 12s

e) SCDNA (extent: North America)

Station-based serially complete datasets (SCDs) of precipitation and temperature observations are important for hydrometeorological studies. Motivated by the lack of serially complete station observations for North America, this study seeks to develop an SCD from 1979 to 2018 from station data. The new SCD for North America (SCDNA) includes daily precipitation, minimum temperature (Tmin), and maximum temperature (Tmax) data for 27 276 stations. Raw meteorological station data were obtained from the Global Historical Climate Network Daily (GHCN-D), the Global Surface Summary of the Day (GSOD), Environment and Climate Change Canada (ECCC), and a compiled station database in Mexico. Stations with at least 8-year-long records were selected, which underwent location correction and were subjected to strict quality control. Outputs from three reanalysis products (ERA5, JRA-55, and MERRA-2) provided auxiliary information to estimate station records

Property	Values
Temporal extent	01/01/1979 – 12/31/2018
Spatial extent	North America : [-177, -52, 7, 83]
Chunks	{‘time’: 1000, ‘ID’: 1000}

ds=cat.atmosphere.scdna.to_dask()
ds

<xarray.Dataset>
Dimensions:    (ID: 27276, time: 14610)
Coordinates:
  * ID         (ID) <U13 'GS91066022701' 'GHMQW00022701' ... 'ECCA008402568'
    elevation  (ID) float32 dask.array<chunksize=(1000,), meta=np.ndarray>
    latitude   (ID) float32 dask.array<chunksize=(1000,), meta=np.ndarray>
    longitude  (ID) float32 dask.array<chunksize=(1000,), meta=np.ndarray>
  * time       (time) datetime64[ns] 1979-01-01 1979-01-02 ... 2018-12-31
Data variables:
    prcp       (ID, time) float32 dask.array<chunksize=(1000, 1000), meta=np.ndarray>
    prcp_flag  (ID, time) float64 dask.array<chunksize=(1000, 1000), meta=np.ndarray>
    prcp_kge   (ID) float32 dask.array<chunksize=(1000,), meta=np.ndarray>
    sflag      (ID) <U3 dask.array<chunksize=(1000,), meta=np.ndarray>
    tmax       (ID, time) float32 dask.array<chunksize=(1000, 1000), meta=np.ndarray>
    tmax_flag  (ID, time) float64 dask.array<chunksize=(1000, 1000), meta=np.ndarray>
    tmax_kge   (ID) float32 dask.array<chunksize=(1000,), meta=np.ndarray>
    tmin       (ID, time) float32 dask.array<chunksize=(1000, 1000), meta=np.ndarray>
    tmin_flag  (ID, time) float64 dask.array<chunksize=(1000, 1000), meta=np.ndarray>
    tmin_kge   (ID) float32 dask.array<chunksize=(1000,), meta=np.ndarray>

xarray.Dataset

• Dimensions:
  • ID: 27276
  • time: 14610
• Coordinates: (5)
  • ID
    (ID)
    <U13
    'GS91066022701' ... 'ECCA008402568'

    array(['GS91066022701', 'GHMQW00022701', 'GHMQC00914490', ..., 'ECCA008402925',
           'ECCA008403669', 'ECCA008402568'], dtype='<U13')

  • elevation
    (ID)
    float32
    dask.array<chunksize=(1000,), meta=np.ndarray>

    Array Chunk Bytes 106.55 kiB 3.91 kiB Shape (27276,) (1000,) Dask graph 28 chunks in 2 graph layers Data type float32 numpy.ndarray

  • latitude
    (ID)
    float32
    dask.array<chunksize=(1000,), meta=np.ndarray>

    Array Chunk Bytes 106.55 kiB 3.91 kiB Shape (27276,) (1000,) Dask graph 28 chunks in 2 graph layers Data type float32 numpy.ndarray

  • longitude
    (ID)
    float32
    dask.array<chunksize=(1000,), meta=np.ndarray>

    Array Chunk Bytes 106.55 kiB 3.91 kiB Shape (27276,) (1000,) Dask graph 28 chunks in 2 graph layers Data type float32 numpy.ndarray

  • time
    (time)
    datetime64[ns]
    1979-01-01 ... 2018-12-31

    array(['1979-01-01T00:00:00.000000000', '1979-01-02T00:00:00.000000000',
           '1979-01-03T00:00:00.000000000', ..., '2018-12-29T00:00:00.000000000',
           '2018-12-30T00:00:00.000000000', '2018-12-31T00:00:00.000000000'],
          dtype='datetime64[ns]')

• Data variables: (10)
  • prcp
    (ID, time)
    float32
    dask.array<chunksize=(1000, 1000), meta=np.ndarray>

    description :

    SCD precipitation (mm/day)

    Array Chunk Bytes 1.48 GiB 3.81 MiB Shape (27276, 14610) (1000, 1000) Dask graph 420 chunks in 2 graph layers Data type float32 numpy.ndarray

  • prcp_flag
    (ID, time)
    float64
    dask.array<chunksize=(1000, 1000), meta=np.ndarray>

    description :

    0: estimates; 1: observation

    Array Chunk Bytes 2.97 GiB 7.63 MiB Shape (27276, 14610) (1000, 1000) Dask graph 420 chunks in 2 graph layers Data type float64 numpy.ndarray

  • prcp_kge
    (ID)
    float32
    dask.array<chunksize=(1000,), meta=np.ndarray>

    description :

    KGE: accuracy metric (-infinity to one)

    Array Chunk Bytes 106.55 kiB 3.91 kiB Shape (27276,) (1000,) Dask graph 28 chunks in 2 graph layers Data type float32 numpy.ndarray

  • sflag
    (ID)
    <U3
    dask.array<chunksize=(1000,), meta=np.ndarray>

    Array Chunk Bytes 319.64 kiB 11.72 kiB Shape (27276,) (1000,) Dask graph 28 chunks in 2 graph layers Data type

  • tmax
    (ID, time)
    float32
    dask.array<chunksize=(1000, 1000), meta=np.ndarray>

    description :

    SCD maximum temperature (Celsius degree)

    Array Chunk Bytes 1.48 GiB 3.81 MiB Shape (27276, 14610) (1000, 1000) Dask graph 420 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tmax_flag
    (ID, time)
    float64
    dask.array<chunksize=(1000, 1000), meta=np.ndarray>

    description :

    0: estimates; 1: observation

    Array Chunk Bytes 2.97 GiB 7.63 MiB Shape (27276, 14610) (1000, 1000) Dask graph 420 chunks in 2 graph layers Data type float64 numpy.ndarray

  • tmax_kge
    (ID)
    float32
    dask.array<chunksize=(1000,), meta=np.ndarray>

    description :

    KGE: accuracy metric (-infinity to one)

    Array Chunk Bytes 106.55 kiB 3.91 kiB Shape (27276,) (1000,) Dask graph 28 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tmin
    (ID, time)
    float32
    dask.array<chunksize=(1000, 1000), meta=np.ndarray>

    description :

    SCD minimum temperature (Celsius degree)

    Array Chunk Bytes 1.48 GiB 3.81 MiB Shape (27276, 14610) (1000, 1000) Dask graph 420 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tmin_flag
    (ID, time)
    float64
    dask.array<chunksize=(1000, 1000), meta=np.ndarray>

    description :

    0: estimates; 1: observation

    Array Chunk Bytes 2.97 GiB 7.63 MiB Shape (27276, 14610) (1000, 1000) Dask graph 420 chunks in 2 graph layers Data type float64 numpy.ndarray

  • tmin_kge
    (ID)
    float32
    dask.array<chunksize=(1000,), meta=np.ndarray>

    description :

    KGE: accuracy metric (-infinity to one)

    Array Chunk Bytes 106.55 kiB 3.91 kiB Shape (27276,) (1000,) Dask graph 28 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (2)
  • ID
    PandasIndex

    PandasIndex(Index(['GS91066022701', 'GHMQW00022701', 'GHMQC00914490', 'GHUSW00025704',
           'GS70454025704', 'GS70204026703', 'GS70203526704', 'GS70203599999',
           'GS70308025713', 'GHUSW00025713',
           ...
           'ECCA008403950', 'ECCA008403045', 'MR71250099999', 'ECCA008403603',
           'MRCA008403600', 'GS71801099999', 'ECCA008403506', 'ECCA008402925',
           'ECCA008403669', 'ECCA008402568'],
          dtype='object', name='ID', length=27276))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1979-01-01', '1979-01-02', '1979-01-03', '1979-01-04',
                   '1979-01-05', '1979-01-06', '1979-01-07', '1979-01-08',
                   '1979-01-09', '1979-01-10',
                   ...
                   '2018-12-22', '2018-12-23', '2018-12-24', '2018-12-25',
                   '2018-12-26', '2018-12-27', '2018-12-28', '2018-12-29',
                   '2018-12-30', '2018-12-31'],
                  dtype='datetime64[ns]', name='time', length=14610, freq=None))

• Attributes: (0)

%%time
ds.prcp \
.sel(time=slice('1996-07-19','1996-07-20')) \
.sum('time') \
.to_dataframe() \
.replace({0:np.nan}) \
.dropna(how='any') \
.hvplot.points(x='longitude',
               y='latitude',
               color='prcp',
               geo=True,
               alpha=0.5,
               xlim=(-180,-30),
               ylim=(0,72),
               tiles='ESRI',
               cmap='gist_ncar',
               clim=(0,100),
               hover_cols=['ID','prcp'],
               width=700,
               height=400,
               title=f'48h precipitation during Saguenay flood event')

CPU times: user 829 ms, sys: 70.1 ms, total: 899 ms
Wall time: 17.4 s

f) 20 Century reanalysis - single levels (extent : Atlantic Northeast)

Using a state-of-the-art data assimilation system and surface pressure observations, the NOAA-CIRES-DOE Twentieth Century Reanalysis (20CR) project has generated a four-dimensional global atmospheric dataset of weather spanning 1836 to 2015 to place current atmospheric circulation patterns into a historical perspective.

Property	Values
Temporal extent:	01/01/1836 – 12/31/2015
Spatial extent:	Atlantic Northeast [-96, -52, 40, 63]
Chunks	{‘time’: 32872, ‘longitude’: 6, ‘latitude’: 3}
Spatial resolution:	1 degrees
Spatial reference:	WGS84 (EPSG:4326)
Temporal resolution:	3 hours
Update frequency:	None

Data access

ds=cat.atmosphere['20_century_reanalysis_single_levels'].to_dask()
ds

<xarray.Dataset>
Dimensions:    (time: 525952, latitude: 24, longitude: 45)
Coordinates:
  * latitude   (latitude) float32 40.0 41.0 42.0 43.0 ... 60.0 61.0 62.0 63.0
  * longitude  (longitude) float32 -96.0 -95.0 -94.0 -93.0 ... -54.0 -53.0 -52.0
  * time       (time) datetime64[ns] 1836-01-01 ... 2015-12-31T21:00:00
Data variables:
    apcp       (time, latitude, longitude) float32 dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>
    cape       (time, latitude, longitude) float32 dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>
    crain      (time, latitude, longitude) float32 dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>
    pr_wtr     (time, latitude, longitude) float32 dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>
    prate      (time, latitude, longitude) float32 dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>
    tcdc       (time, latitude, longitude) float32 dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>
    tmax       (time, latitude, longitude) float32 dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>
    tmin       (time, latitude, longitude) float32 dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>
Attributes: (12/24)
    Conventions:               CF-1.2
    References:                https://www.psl.noaa.gov/data/gridded/data.20t...
    assimilation_algorithm:    Ensemble Kalman Filter with 4DIAU
    citation:                  Compo,G.P. <https://www.psl.noaa.gov/people/gi...
    citation1:                 Slivinski, L. C, G. P. Compo, J. S. Whitaker, ...
    comments:                  Data are from \nNOAA/CIRES/DOE 20th Century Re...
    ...                        ...
    product:                   reanalysis
    source:                    20CRv3si 2018, Ensemble Kalman Filter, ocean (...
    spatial_resolution:        1.0 degree
    standard_name_vocabulary:  NetCDF Climate and Forecast (CF) Metadata Conv...
    title:                     8x Daily NOAA/CIRES/DOE 20th Century Reanalysi...
    version:                   3si

xarray.Dataset

• Dimensions:
  • time: 525952
  • latitude: 24
  • longitude: 45
• Coordinates: (3)
  • latitude
    (latitude)
    float32
    40.0 41.0 42.0 ... 61.0 62.0 63.0

    actual_range :

    [90.0, -90.0]

    axis :

    Y

    coordinate_defines :

    point

    long_name :

    Latitude

    standard_name :

    latitude

    units :

    degrees_north

    array([40., 41., 42., 43., 44., 45., 46., 47., 48., 49., 50., 51., 52., 53.,
           54., 55., 56., 57., 58., 59., 60., 61., 62., 63.], dtype=float32)

  • longitude
    (longitude)
    float32
    -96.0 -95.0 -94.0 ... -53.0 -52.0

    array([-96., -95., -94., -93., -92., -91., -90., -89., -88., -87., -86., -85.,
           -84., -83., -82., -81., -80., -79., -78., -77., -76., -75., -74., -73.,
           -72., -71., -70., -69., -68., -67., -66., -65., -64., -63., -62., -61.,
           -60., -59., -58., -57., -56., -55., -54., -53., -52.], dtype=float32)

  • time
    (time)
    datetime64[ns]
    1836-01-01 ... 2015-12-31T21:00:00

    axis :

    T

    coordinate_defines :

    start

    delta_t :

    0000-00-00 03:00:00

    long_name :

    Time

    standard_name :

    time

    array(['1836-01-01T00:00:00.000000000', '1836-01-01T03:00:00.000000000',
           '1836-01-01T06:00:00.000000000', ..., '2015-12-31T15:00:00.000000000',
           '2015-12-31T18:00:00.000000000', '2015-12-31T21:00:00.000000000'],
          dtype='datetime64[ns]')

• Data variables: (8)
  • apcp
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>

    GRIB_name :

    APCP

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [0.0, 14.399999618530273]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Surface

    long_name :

    3-hourly accumulated total precipitation at surface

    parent_stat :

    Other

    standard_name :

    precipitation_amount

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    kg/m^2

    valid_range :

    [0.0, 100.0]

    var_desc :

    Precipitation amount

    Array Chunk Bytes 2.12 GiB 2.26 MiB Shape (525952, 24, 45) (32872, 3, 6) Dask graph 1024 chunks in 2 graph layers Data type float32 numpy.ndarray

  • cape
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>

    GRIB_name :

    CAPE

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [0.0, 3266.0]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Surface

    long_name :

    3-hourly CAPE

    parent_stat :

    Other

    standard_name :

    atmosphere_convective_available_potential_energy

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    J/kg

    valid_range :

    [0.0, 8000.0]

    var_desc :

    convective available potential energy

    Array Chunk Bytes 2.12 GiB 2.26 MiB Shape (525952, 24, 45) (32872, 3, 6) Dask graph 1024 chunks in 2 graph layers Data type float32 numpy.ndarray

  • crain
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>

    GRIB_name :

    CRAIN

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [0.0, 1.0]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Surface

    long_name :

    3-hourly Categorical Rain at surface

    parent_stat :

    Other

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    1

    valid_range :

    [0.0, 1.0]

    var_desc :

    categorical rain

    Array Chunk Bytes 2.12 GiB 2.26 MiB Shape (525952, 24, 45) (32872, 3, 6) Dask graph 1024 chunks in 2 graph layers Data type float32 numpy.ndarray

  • pr_wtr
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>

    GRIB_name :

    PWAT

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [0.09999999403953552, 71.4000015258789]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Entire Atmosphere Considered As a Single Layer

    long_name :

    3-hourly Precipitable Water for entire atmosphere

    parent_stat :

    Other

    standard_name :

    atmosphere_water_vapor_content

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    kg/m^2

    valid_range :

    [-50.0, 150.0]

    var_desc :

    Precipitable Water Content

    Array Chunk Bytes 2.12 GiB 2.26 MiB Shape (525952, 24, 45) (32872, 3, 6) Dask graph 1024 chunks in 2 graph layers Data type float32 numpy.ndarray

  • prate
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>

    GRIB_name :

    PRATE

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [0.0, 0.0013289999915286899]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Surface

    long_name :

    3-hourly Precipitation Rate at surface

    parent_stat :

    Other

    standard_name :

    precipitation_flux

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    kg/m^2/s

    valid_range :

    [0.0, 0.006552999839186668]

    var_desc :

    Precipitation Rate

    Array Chunk Bytes 2.12 GiB 2.26 MiB Shape (525952, 24, 45) (32872, 3, 6) Dask graph 1024 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tcdc
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>

    GRIB_name :

    TCDC

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [0.0, 100.48970031738281]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Entire Atmosphere Considered As a Single Layer

    long_name :

    3-hourly Total Cloud Cover

    parent_stat :

    Other

    standard_name :

    cloud_area_fraction

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    %

    valid_range :

    [0.0, 100.0]

    var_desc :

    Total cloud cover

    Array Chunk Bytes 2.12 GiB 2.26 MiB Shape (525952, 24, 45) (32872, 3, 6) Dask graph 1024 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tmax
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>

    GRIB_name :

    TMAX

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [215.9904022216797, 322.3743591308594]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    2 m

    long_name :

    3-hourly Maximum Temperature at 2 m

    parent_stat :

    Other

    standard_name :

    air_temperature

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    degK

    valid_range :

    [100.0, 500.0]

    var_desc :

    Maximum Temperature

    Array Chunk Bytes 2.12 GiB 2.26 MiB Shape (525952, 24, 45) (32872, 3, 6) Dask graph 1024 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tmin
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(32872, 3, 6), meta=np.ndarray>

    GRIB_name :

    TMIN

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [214.71347045898438, 321.08837890625]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    2 m

    long_name :

    3-hourly Minimum Temperature at 2 m

    parent_stat :

    Other

    standard_name :

    air_temperature

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    degK

    valid_range :

    [100.0, 500.0]

    var_desc :

    Minimum Temperature

    Array Chunk Bytes 2.12 GiB 2.26 MiB Shape (525952, 24, 45) (32872, 3, 6) Dask graph 1024 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (3)
  • latitude
    PandasIndex

    PandasIndex(Float64Index([40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0,
                  51.0, 52.0, 53.0, 54.0, 55.0, 56.0, 57.0, 58.0, 59.0, 60.0, 61.0,
                  62.0, 63.0],
                 dtype='float64', name='latitude'))

  • longitude
    PandasIndex

    PandasIndex(Float64Index([-96.0, -95.0, -94.0, -93.0, -92.0, -91.0, -90.0, -89.0, -88.0,
                  -87.0, -86.0, -85.0, -84.0, -83.0, -82.0, -81.0, -80.0, -79.0,
                  -78.0, -77.0, -76.0, -75.0, -74.0, -73.0, -72.0, -71.0, -70.0,
                  -69.0, -68.0, -67.0, -66.0, -65.0, -64.0, -63.0, -62.0, -61.0,
                  -60.0, -59.0, -58.0, -57.0, -56.0, -55.0, -54.0, -53.0, -52.0],
                 dtype='float64', name='longitude'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1836-01-01 00:00:00', '1836-01-01 03:00:00',
                   '1836-01-01 06:00:00', '1836-01-01 09:00:00',
                   '1836-01-01 12:00:00', '1836-01-01 15:00:00',
                   '1836-01-01 18:00:00', '1836-01-01 21:00:00',
                   '1836-01-02 00:00:00', '1836-01-02 03:00:00',
                   ...
                   '2015-12-30 18:00:00', '2015-12-30 21:00:00',
                   '2015-12-31 00:00:00', '2015-12-31 03:00:00',
                   '2015-12-31 06:00:00', '2015-12-31 09:00:00',
                   '2015-12-31 12:00:00', '2015-12-31 15:00:00',
                   '2015-12-31 18:00:00', '2015-12-31 21:00:00'],
                  dtype='datetime64[ns]', name='time', length=525952, freq=None))

• Attributes: (24)

  Conventions :

  CF-1.2

  References :

  https://www.psl.noaa.gov/data/gridded/data.20thC_ReanV3.html

  assimilation_algorithm :

  Ensemble Kalman Filter with 4DIAU

  citation :

  Compo,G.P. <https://www.psl.noaa.gov/people/gilbert.p.compo>, J.S. Whitaker <https://www.psl.noaa.gov/people/jeffrey.s.whitaker>, P.D. Sardeshmukh, N.Matsui, R.J. Allan, X. Yin, B.E. Gleason, R.S. Vose, G. Rutledge, P. Bessemoulin, S. Bronnimann, M. Brunet, R.I. Crouthamel, A.N. Grant, P.Y. Groisman, P.D. Jones, M. Kruk, A.C. Kruger, G.J. Marshal, M. Maugeri, H.Y. Mok, O. Nordli, T.F. Ross, R.M. Trio, X.L. Wang, S.D. Woodruff, and S.J. Worley, 2011: The Twentieth Century Reanalysis Project <http://onlinelibrary.wiley.com/doi/10.1002/qj.776/abstract>. Quarterly J. Roy. Meteorol. Soc./, 137, 1-28. DOI: 10.1002/qj.776.

  citation1 :

  Slivinski, L. C, G. P. Compo, J. S. Whitaker, P. D. Sardeshmukh, B. S. Giese, C. McColl, R. Allan, X. Yin, R. Vose, H. Titchner, J. Kennedy, L. J. Spencer, L. Ashcroft, S. Bronnimann, M. Brunet, D. Camuffo, R. Cornes, T. A. Cram, R. Crouthamel, F. Dominguez-Castro, J. E. Freeman, J. Gergis, E. Hawkins, P. D. Jones, S. Jourdain, A. Kaplan, H. Kubota, F. Le Blancq, T.-C. Lee, A. Lorrey, J. Luterbacher, M. Maugeri, C. J. Mock, G.W. K. Moore, R. Przybylak, C. Pudmenzky, C. Reason, V. C. Slonosky, C. Smith, B. Tinz, B. Trewin, M. A. Valente, X. L. Wang, C. Wilkinson, K. Wood, and P. Wyszynski: Towards a more reliable historical reanalysis: Improvements for version 3 of the Twentieth Century Reanalysis System. Quarterly Journal of the Royal Meteorological Society. 2019; 145: 2876- 2908. doi:10.1002/qj.3598.

  comments :

  Data are from NOAA/CIRES/DOE 20th Century Reanalysis version 3 (8x/day). It consists of most variables interpolated to pressure surfaces from model hybrid (sigma/pressure) surfaces.

  contact :

  psl.data@noaa.gov

  creation_date :

  2019-03-25T20:56:06Z

  creator_name :

  NOAA/PSL

  dataset_title :

  NOAA/CIRES/DOE 20th Century Reanalysis (V3)

  forcing :

  CO2, Sl, Vl, SST, Oz (CO2 as in Saha et al (2010), Oz is prescribed from CMIP5, SST is SODAsi.3, Sl is Total Solar Irradiance (TSI) Reconstruction based on NRLTSI2, Vl defined as in Crowley & Unterman 2013)

  forcing_note :

  Additional information on the external forcings uses in this reanalysis can be found in Slivinski et al. (2019) (citation1)

  history :

  created 2019/03 by Hoop (chunked, deflated non-packed netCDF4-Classic) esrl.psd.data@noaa.gov

  institution :

  NOAA Physical Sciences Laboratory & CU/CIRES (325 Broadway R/PSL, Boulder, CO 80305)

  license :

  These data are available free of charge under the PSD data policy

  name_of_model_used :

  ensda_v451

  observations :

  International Surface Pressure Databank version 4.7

  platform :

  Model

  product :

  reanalysis

  source :

  20CRv3si 2018, Ensemble Kalman Filter, ocean (specified): SODAsi.3; atmosphere: NCEP GFS v14.0.1; sea ice (specified): HadISST2.3 (years prior to 1850 use climatology of 1860-1891); land: Noah 4 layer version 2

  spatial_resolution :

  1.0 degree

  standard_name_vocabulary :

  NetCDF Climate and Forecast (CF) Metadata Convention

  title :

  8x Daily NOAA/CIRES/DOE 20th Century Reanalysis V3

  version :

  3si

Working with the data

Here we compute a simple line plot :

%%time
ds.sel(latitude=45,
       longitude=-75) \
.prate \
.hvplot(grid=True)

CPU times: user 120 ms, sys: 6.36 ms, total: 127 ms
Wall time: 1.56 s

g) 20 Century reanalysis - single levels (large area : for analysis in space)

Using a state-of-the-art data assimilation system and surface pressure observations, the NOAA-CIRES-DOE Twentieth Century Reanalysis (20CR) project has generated a four-dimensional global atmospheric dataset of weather spanning 1836 to 2015 to place current atmospheric circulation patterns into a historical perspective.

Property	Values
Temporal extent:	01/01/1836 – 12/31/2015
Spatial extent:	Atlantic Northeast [-96, -52, 40, 63]
Chunks	{‘time’: 100, ‘longitude’: 45, ‘latitude’: 24}
Spatial resolution:	1 degrees
Spatial reference:	WGS84 (EPSG:4326)
Temporal resolution:	3 hours
Update frequency:	None

ds=cat.atmosphere['20_century_reanalysis_single_levels_large_area'].to_dask()
ds

<xarray.Dataset>
Dimensions:    (time: 525952, latitude: 24, longitude: 45)
Coordinates:
  * latitude   (latitude) float32 40.0 41.0 42.0 43.0 ... 60.0 61.0 62.0 63.0
  * longitude  (longitude) float32 -96.0 -95.0 -94.0 -93.0 ... -54.0 -53.0 -52.0
  * time       (time) datetime64[ns] 1836-01-01 ... 2015-12-31T21:00:00
Data variables:
    apcp       (time, latitude, longitude) float32 dask.array<chunksize=(100, 24, 45), meta=np.ndarray>
    cape       (time, latitude, longitude) float32 dask.array<chunksize=(100, 24, 45), meta=np.ndarray>
    crain      (time, latitude, longitude) float32 dask.array<chunksize=(100, 24, 45), meta=np.ndarray>
    pr_wtr     (time, latitude, longitude) float32 dask.array<chunksize=(100, 24, 45), meta=np.ndarray>
    prate      (time, latitude, longitude) float32 dask.array<chunksize=(100, 24, 45), meta=np.ndarray>
    tcdc       (time, latitude, longitude) float32 dask.array<chunksize=(100, 24, 45), meta=np.ndarray>
    tmax       (time, latitude, longitude) float32 dask.array<chunksize=(100, 24, 45), meta=np.ndarray>
    tmin       (time, latitude, longitude) float32 dask.array<chunksize=(100, 24, 45), meta=np.ndarray>
Attributes: (12/24)
    Conventions:               CF-1.2
    References:                https://www.psl.noaa.gov/data/gridded/data.20t...
    assimilation_algorithm:    Ensemble Kalman Filter with 4DIAU
    citation:                  Compo,G.P. <https://www.psl.noaa.gov/people/gi...
    citation1:                 Slivinski, L. C, G. P. Compo, J. S. Whitaker, ...
    comments:                  Data are from \nNOAA/CIRES/DOE 20th Century Re...
    ...                        ...
    product:                   reanalysis
    source:                    20CRv3si 2018, Ensemble Kalman Filter, ocean (...
    spatial_resolution:        1.0 degree
    standard_name_vocabulary:  NetCDF Climate and Forecast (CF) Metadata Conv...
    title:                     8x Daily NOAA/CIRES/DOE 20th Century Reanalysi...
    version:                   3si

xarray.Dataset

• Dimensions:
  • time: 525952
  • latitude: 24
  • longitude: 45
• Coordinates: (3)
  • latitude
    (latitude)
    float32
    40.0 41.0 42.0 ... 61.0 62.0 63.0

    actual_range :

    [90.0, -90.0]

    axis :

    Y

    coordinate_defines :

    point

    long_name :

    Latitude

    standard_name :

    latitude

    units :

    degrees_north

    array([40., 41., 42., 43., 44., 45., 46., 47., 48., 49., 50., 51., 52., 53.,
           54., 55., 56., 57., 58., 59., 60., 61., 62., 63.], dtype=float32)

  • longitude
    (longitude)
    float32
    -96.0 -95.0 -94.0 ... -53.0 -52.0

    array([-96., -95., -94., -93., -92., -91., -90., -89., -88., -87., -86., -85.,
           -84., -83., -82., -81., -80., -79., -78., -77., -76., -75., -74., -73.,
           -72., -71., -70., -69., -68., -67., -66., -65., -64., -63., -62., -61.,
           -60., -59., -58., -57., -56., -55., -54., -53., -52.], dtype=float32)

  • time
    (time)
    datetime64[ns]
    1836-01-01 ... 2015-12-31T21:00:00

    axis :

    T

    coordinate_defines :

    start

    delta_t :

    0000-00-00 03:00:00

    long_name :

    Time

    standard_name :

    time

    array(['1836-01-01T00:00:00.000000000', '1836-01-01T03:00:00.000000000',
           '1836-01-01T06:00:00.000000000', ..., '2015-12-31T15:00:00.000000000',
           '2015-12-31T18:00:00.000000000', '2015-12-31T21:00:00.000000000'],
          dtype='datetime64[ns]')

• Data variables: (8)
  • apcp
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(100, 24, 45), meta=np.ndarray>

    GRIB_name :

    APCP

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [0.0, 14.399999618530273]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Surface

    long_name :

    3-hourly accumulated total precipitation at surface

    parent_stat :

    Other

    standard_name :

    precipitation_amount

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    kg/m^2

    valid_range :

    [0.0, 100.0]

    var_desc :

    Precipitation amount

    Array Chunk Bytes 2.12 GiB 421.88 kiB Shape (525952, 24, 45) (100, 24, 45) Dask graph 5260 chunks in 2 graph layers Data type float32 numpy.ndarray

  • cape
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(100, 24, 45), meta=np.ndarray>

    GRIB_name :

    CAPE

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [0.0, 3266.0]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Surface

    long_name :

    3-hourly CAPE

    parent_stat :

    Other

    standard_name :

    atmosphere_convective_available_potential_energy

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    J/kg

    valid_range :

    [0.0, 8000.0]

    var_desc :

    convective available potential energy

    Array Chunk Bytes 2.12 GiB 421.88 kiB Shape (525952, 24, 45) (100, 24, 45) Dask graph 5260 chunks in 2 graph layers Data type float32 numpy.ndarray

  • crain
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(100, 24, 45), meta=np.ndarray>

    GRIB_name :

    CRAIN

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [0.0, 1.0]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Surface

    long_name :

    3-hourly Categorical Rain at surface

    parent_stat :

    Other

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    1

    valid_range :

    [0.0, 1.0]

    var_desc :

    categorical rain

    Array Chunk Bytes 2.12 GiB 421.88 kiB Shape (525952, 24, 45) (100, 24, 45) Dask graph 5260 chunks in 2 graph layers Data type float32 numpy.ndarray

  • pr_wtr
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(100, 24, 45), meta=np.ndarray>

    GRIB_name :

    PWAT

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [0.09999999403953552, 71.4000015258789]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Entire Atmosphere Considered As a Single Layer

    long_name :

    3-hourly Precipitable Water for entire atmosphere

    parent_stat :

    Other

    standard_name :

    atmosphere_water_vapor_content

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    kg/m^2

    valid_range :

    [-50.0, 150.0]

    var_desc :

    Precipitable Water Content

    Array Chunk Bytes 2.12 GiB 421.88 kiB Shape (525952, 24, 45) (100, 24, 45) Dask graph 5260 chunks in 2 graph layers Data type float32 numpy.ndarray

  • prate
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(100, 24, 45), meta=np.ndarray>

    GRIB_name :

    PRATE

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [0.0, 0.0013289999915286899]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Surface

    long_name :

    3-hourly Precipitation Rate at surface

    parent_stat :

    Other

    standard_name :

    precipitation_flux

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    kg/m^2/s

    valid_range :

    [0.0, 0.006552999839186668]

    var_desc :

    Precipitation Rate

    Array Chunk Bytes 2.12 GiB 421.88 kiB Shape (525952, 24, 45) (100, 24, 45) Dask graph 5260 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tcdc
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(100, 24, 45), meta=np.ndarray>

    GRIB_name :

    TCDC

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [0.0, 100.48970031738281]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Entire Atmosphere Considered As a Single Layer

    long_name :

    3-hourly Total Cloud Cover

    parent_stat :

    Other

    standard_name :

    cloud_area_fraction

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    %

    valid_range :

    [0.0, 100.0]

    var_desc :

    Total cloud cover

    Array Chunk Bytes 2.12 GiB 421.88 kiB Shape (525952, 24, 45) (100, 24, 45) Dask graph 5260 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tmax
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(100, 24, 45), meta=np.ndarray>

    GRIB_name :

    TMAX

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [215.9904022216797, 322.3743591308594]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    2 m

    long_name :

    3-hourly Maximum Temperature at 2 m

    parent_stat :

    Other

    standard_name :

    air_temperature

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    degK

    valid_range :

    [100.0, 500.0]

    var_desc :

    Maximum Temperature

    Array Chunk Bytes 2.12 GiB 421.88 kiB Shape (525952, 24, 45) (100, 24, 45) Dask graph 5260 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tmin
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(100, 24, 45), meta=np.ndarray>

    GRIB_name :

    TMIN

    GridType :

    Cylindrical Equidistant Projection Grid

    actual_range :

    [214.71347045898438, 321.08837890625]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    2 m

    long_name :

    3-hourly Minimum Temperature at 2 m

    parent_stat :

    Other

    standard_name :

    air_temperature

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    degK

    valid_range :

    [100.0, 500.0]

    var_desc :

    Minimum Temperature

    Array Chunk Bytes 2.12 GiB 421.88 kiB Shape (525952, 24, 45) (100, 24, 45) Dask graph 5260 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (3)
  • latitude
    PandasIndex

    PandasIndex(Float64Index([40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0,
                  51.0, 52.0, 53.0, 54.0, 55.0, 56.0, 57.0, 58.0, 59.0, 60.0, 61.0,
                  62.0, 63.0],
                 dtype='float64', name='latitude'))

  • longitude
    PandasIndex

    PandasIndex(Float64Index([-96.0, -95.0, -94.0, -93.0, -92.0, -91.0, -90.0, -89.0, -88.0,
                  -87.0, -86.0, -85.0, -84.0, -83.0, -82.0, -81.0, -80.0, -79.0,
                  -78.0, -77.0, -76.0, -75.0, -74.0, -73.0, -72.0, -71.0, -70.0,
                  -69.0, -68.0, -67.0, -66.0, -65.0, -64.0, -63.0, -62.0, -61.0,
                  -60.0, -59.0, -58.0, -57.0, -56.0, -55.0, -54.0, -53.0, -52.0],
                 dtype='float64', name='longitude'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1836-01-01 00:00:00', '1836-01-01 03:00:00',
                   '1836-01-01 06:00:00', '1836-01-01 09:00:00',
                   '1836-01-01 12:00:00', '1836-01-01 15:00:00',
                   '1836-01-01 18:00:00', '1836-01-01 21:00:00',
                   '1836-01-02 00:00:00', '1836-01-02 03:00:00',
                   ...
                   '2015-12-30 18:00:00', '2015-12-30 21:00:00',
                   '2015-12-31 00:00:00', '2015-12-31 03:00:00',
                   '2015-12-31 06:00:00', '2015-12-31 09:00:00',
                   '2015-12-31 12:00:00', '2015-12-31 15:00:00',
                   '2015-12-31 18:00:00', '2015-12-31 21:00:00'],
                  dtype='datetime64[ns]', name='time', length=525952, freq=None))

• Attributes: (24)

  Conventions :

  CF-1.2

  References :

  https://www.psl.noaa.gov/data/gridded/data.20thC_ReanV3.html

  assimilation_algorithm :

  Ensemble Kalman Filter with 4DIAU

  citation :

  Compo,G.P. <https://www.psl.noaa.gov/people/gilbert.p.compo>, J.S. Whitaker <https://www.psl.noaa.gov/people/jeffrey.s.whitaker>, P.D. Sardeshmukh, N.Matsui, R.J. Allan, X. Yin, B.E. Gleason, R.S. Vose, G. Rutledge, P. Bessemoulin, S. Bronnimann, M. Brunet, R.I. Crouthamel, A.N. Grant, P.Y. Groisman, P.D. Jones, M. Kruk, A.C. Kruger, G.J. Marshal, M. Maugeri, H.Y. Mok, O. Nordli, T.F. Ross, R.M. Trio, X.L. Wang, S.D. Woodruff, and S.J. Worley, 2011: The Twentieth Century Reanalysis Project <http://onlinelibrary.wiley.com/doi/10.1002/qj.776/abstract>. Quarterly J. Roy. Meteorol. Soc./, 137, 1-28. DOI: 10.1002/qj.776.

  citation1 :

  Slivinski, L. C, G. P. Compo, J. S. Whitaker, P. D. Sardeshmukh, B. S. Giese, C. McColl, R. Allan, X. Yin, R. Vose, H. Titchner, J. Kennedy, L. J. Spencer, L. Ashcroft, S. Bronnimann, M. Brunet, D. Camuffo, R. Cornes, T. A. Cram, R. Crouthamel, F. Dominguez-Castro, J. E. Freeman, J. Gergis, E. Hawkins, P. D. Jones, S. Jourdain, A. Kaplan, H. Kubota, F. Le Blancq, T.-C. Lee, A. Lorrey, J. Luterbacher, M. Maugeri, C. J. Mock, G.W. K. Moore, R. Przybylak, C. Pudmenzky, C. Reason, V. C. Slonosky, C. Smith, B. Tinz, B. Trewin, M. A. Valente, X. L. Wang, C. Wilkinson, K. Wood, and P. Wyszynski: Towards a more reliable historical reanalysis: Improvements for version 3 of the Twentieth Century Reanalysis System. Quarterly Journal of the Royal Meteorological Society. 2019; 145: 2876- 2908. doi:10.1002/qj.3598.

  comments :

  Data are from NOAA/CIRES/DOE 20th Century Reanalysis version 3 (8x/day). It consists of most variables interpolated to pressure surfaces from model hybrid (sigma/pressure) surfaces.

  contact :

  psl.data@noaa.gov

  creation_date :

  2019-03-25T20:56:06Z

  creator_name :

  NOAA/PSL

  dataset_title :

  NOAA/CIRES/DOE 20th Century Reanalysis (V3)

  forcing :

  CO2, Sl, Vl, SST, Oz (CO2 as in Saha et al (2010), Oz is prescribed from CMIP5, SST is SODAsi.3, Sl is Total Solar Irradiance (TSI) Reconstruction based on NRLTSI2, Vl defined as in Crowley & Unterman 2013)

  forcing_note :

  Additional information on the external forcings uses in this reanalysis can be found in Slivinski et al. (2019) (citation1)

  history :

  created 2019/03 by Hoop (chunked, deflated non-packed netCDF4-Classic) esrl.psd.data@noaa.gov

  institution :

  NOAA Physical Sciences Laboratory & CU/CIRES (325 Broadway R/PSL, Boulder, CO 80305)

  license :

  These data are available free of charge under the PSD data policy

  name_of_model_used :

  ensda_v451

  observations :

  International Surface Pressure Databank version 4.7

  platform :

  Model

  product :

  reanalysis

  source :

  20CRv3si 2018, Ensemble Kalman Filter, ocean (specified): SODAsi.3; atmosphere: NCEP GFS v14.0.1; sea ice (specified): HadISST2.3 (years prior to 1850 use climatology of 1860-1891); land: Noah 4 layer version 2

  spatial_resolution :

  1.0 degree

  standard_name_vocabulary :

  NetCDF Climate and Forecast (CF) Metadata Convention

  title :

  8x Daily NOAA/CIRES/DOE 20th Century Reanalysis V3

  version :

  3si

Working with the data

%%time
ds.sel(time='2000-01-01T00:00') \
.tmax \
.hvplot(grid=True,
        cmap='cwr',
        geo=True,
        tiles='CartoLight',
        alpha=0.75,
        width=700,
        height=400,)

CPU times: user 76.9 ms, sys: 3.56 ms, total: 80.5 ms
Wall time: 78.5 ms

Other datasets :

The previous examples can be applied to the following datasets as well. We will let the end user experiment with them!

ds=cat.atmosphere['20_century_reanalysis_pressure_levels'].to_dask()
ds

<xarray.Dataset>
Dimensions:    (time: 525952, level: 17, latitude: 24, longitude: 45)
Coordinates:
  * latitude   (latitude) float32 40.0 41.0 42.0 43.0 ... 60.0 61.0 62.0 63.0
  * level      (level) float64 1.0 5.0 10.0 20.0 ... 700.0 800.0 900.0 1e+03
  * longitude  (longitude) float32 -96.0 -95.0 -94.0 -93.0 ... -54.0 -53.0 -52.0
  * time       (time) datetime64[ns] 1836-01-01 ... 2015-12-31T21:00:00
Data variables:
    air        (time, level, latitude, longitude) float32 dask.array<chunksize=(29200, 1, 24, 25), meta=np.ndarray>
    hgt        (time, level, latitude, longitude) float32 dask.array<chunksize=(29200, 1, 24, 25), meta=np.ndarray>
    omega      (time, level, latitude, longitude) float32 dask.array<chunksize=(29200, 1, 24, 25), meta=np.ndarray>
    rhum       (time, level, latitude, longitude) float32 dask.array<chunksize=(29200, 1, 24, 25), meta=np.ndarray>
Attributes: (12/25)
    Conventions:                     CF-1.2
    DODS_EXTRA.Unlimited_Dimension:  time
    References:                      https://www.esrl.noaa.gov/psd/data/gridd...
    assimilation_algorithm:          Ensemble Kalman Filter with 4DIAU
    citation:                        Compo,G.P. <https://www.esrl.noaa.gov/ps...
    citation1:                       Slivinski, L. C, G. P. Compo, J. S. Whit...
    ...                              ...
    product:                         reanalysis
    source:                          20CRv3si 2018, Ensemble Kalman Filter, o...
    spatial_resolution:              1.0 degree
    standard_name_vocabulary:        NetCDF Climate and Forecast (CF) Metadat...
    title:                           8x Daily NOAA/CIRES/DOE 20th Century Rea...
    version:                         3si

xarray.Dataset

• Dimensions:
  • time: 525952
  • level: 17
  • latitude: 24
  • longitude: 45
• Coordinates: (4)
  • latitude
    (latitude)
    float32
    40.0 41.0 42.0 ... 61.0 62.0 63.0

    actual_range :

    [90.0, -90.0]

    axis :

    Y

    coordinate_defines :

    point

    long_name :

    Latitude

    standard_name :

    latitude

    units :

    degrees_north

    array([40., 41., 42., 43., 44., 45., 46., 47., 48., 49., 50., 51., 52., 53.,
           54., 55., 56., 57., 58., 59., 60., 61., 62., 63.], dtype=float32)

  • level
    (level)
    float64
    1.0 5.0 10.0 ... 800.0 900.0 1e+03

    actual_range :

    [1000.0, 1.0]

    axis :

    Z

    coordinate_defines :

    point

    long_name :

    Level

    positive :

    down

    units :

    millibar

    array([   1.,    5.,   10.,   20.,   30.,   50.,   70.,  100.,  200.,  300.,
            400.,  500.,  600.,  700.,  800.,  900., 1000.])

  • longitude
    (longitude)
    float32
    -96.0 -95.0 -94.0 ... -53.0 -52.0

    array([-96., -95., -94., -93., -92., -91., -90., -89., -88., -87., -86., -85.,
           -84., -83., -82., -81., -80., -79., -78., -77., -76., -75., -74., -73.,
           -72., -71., -70., -69., -68., -67., -66., -65., -64., -63., -62., -61.,
           -60., -59., -58., -57., -56., -55., -54., -53., -52.], dtype=float32)

  • time
    (time)
    datetime64[ns]
    1836-01-01 ... 2015-12-31T21:00:00

    _ChunkSizes :

    1

    axis :

    T

    coordinate_defines :

    point

    delta_t :

    0000-00-00 03:00:00

    long_name :

    Time

    standard_name :

    time

    array(['1836-01-01T00:00:00.000000000', '1836-01-01T03:00:00.000000000',
           '1836-01-01T06:00:00.000000000', ..., '2015-12-31T15:00:00.000000000',
           '2015-12-31T18:00:00.000000000', '2015-12-31T21:00:00.000000000'],
          dtype='datetime64[ns]')

• Data variables: (4)
  • air
    (time, level, latitude, longitude)
    float32
    dask.array<chunksize=(29200, 1, 24, 25), meta=np.ndarray>

    GRIB_name :

    TMP

    GridType :

    Cylindrical Equidistant Projection Grid

    _ChunkSizes :

    [1, 1, 181, 360]

    actual_range :

    [188.0, 321.8634338378906]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Pressure Levels

    long_name :

    3-hourly Air Temperature on Pressure Levels

    parent_stat :

    Other

    standard_name :

    air_temperature

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    degK

    valid_range :

    [100.0, 500.0]

    var_desc :

    Air temperature

    Array Chunk Bytes 35.97 GiB 66.83 MiB Shape (525952, 17, 24, 45) (29200, 1, 24, 25) Dask graph 646 chunks in 2 graph layers Data type float32 numpy.ndarray

  • hgt
    (time, level, latitude, longitude)
    float32
    dask.array<chunksize=(29200, 1, 24, 25), meta=np.ndarray>

    GRIB_name :

    HGT

    GridType :

    Cylindrical Equidistant Projection Grid

    _ChunkSizes :

    [1, 1, 181, 360]

    actual_range :

    [-283.2093200683594, 50456.0703125]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Pressure Levels

    long_name :

    3-hourly Geopotential Height on Pressure Levels

    parent_stat :

    Other

    standard_name :

    geopotential_height

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    m

    valid_range :

    [-1500.0, 60000.0]

    var_desc :

    Geopotential height

    Array Chunk Bytes 35.97 GiB 66.83 MiB Shape (525952, 17, 24, 45) (29200, 1, 24, 25) Dask graph 646 chunks in 2 graph layers Data type float32 numpy.ndarray

  • omega
    (time, level, latitude, longitude)
    float32
    dask.array<chunksize=(29200, 1, 24, 25), meta=np.ndarray>

    GRIB_name :

    VVEL

    GridType :

    Cylindrical Equidistant Projection Grid

    _ChunkSizes :

    [1, 1, 181, 360]

    actual_range :

    [-1.5944035053253174, 2.981666326522827]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Pressure Levels

    long_name :

    3-hourly Omega on Pressure Levels

    parent_stat :

    Other

    standard_name :

    vertical_air_velocity_expressed_as_tendency_of_pressure

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    Pascal/s

    valid_range :

    [-15.0, 15.0]

    var_desc :

    Omega (dp/dt)

    Array Chunk Bytes 35.97 GiB 66.83 MiB Shape (525952, 17, 24, 45) (29200, 1, 24, 25) Dask graph 646 chunks in 2 graph layers Data type float32 numpy.ndarray

  • rhum
    (time, level, latitude, longitude)
    float32
    dask.array<chunksize=(29200, 1, 24, 25), meta=np.ndarray>

    GRIB_name :

    RH

    GridType :

    Cylindrical Equidistant Projection Grid

    _ChunkSizes :

    [1, 1, 181, 360]

    actual_range :

    [0.0, 100.0]

    comment :

    Calculated from the ensemble mean and not averaged from the relative humidity of individual ensemble members. The values are similar but not exact

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Pressure Levels

    long_name :

    3-hourly Relative Humidity on Pressure Levels

    parent_stat :

    Other

    standard_name :

    relative_humidity

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    %

    valid_range :

    [0.0, 100.0]

    var_desc :

    Relative Humidity

    Array Chunk Bytes 35.97 GiB 66.83 MiB Shape (525952, 17, 24, 45) (29200, 1, 24, 25) Dask graph 646 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (4)
  • latitude
    PandasIndex

    PandasIndex(Float64Index([40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0,
                  51.0, 52.0, 53.0, 54.0, 55.0, 56.0, 57.0, 58.0, 59.0, 60.0, 61.0,
                  62.0, 63.0],
                 dtype='float64', name='latitude'))

  • level
    PandasIndex

    PandasIndex(Float64Index([   1.0,    5.0,   10.0,   20.0,   30.0,   50.0,   70.0,  100.0,
                   200.0,  300.0,  400.0,  500.0,  600.0,  700.0,  800.0,  900.0,
                  1000.0],
                 dtype='float64', name='level'))

  • longitude
    PandasIndex

    PandasIndex(Float64Index([-96.0, -95.0, -94.0, -93.0, -92.0, -91.0, -90.0, -89.0, -88.0,
                  -87.0, -86.0, -85.0, -84.0, -83.0, -82.0, -81.0, -80.0, -79.0,
                  -78.0, -77.0, -76.0, -75.0, -74.0, -73.0, -72.0, -71.0, -70.0,
                  -69.0, -68.0, -67.0, -66.0, -65.0, -64.0, -63.0, -62.0, -61.0,
                  -60.0, -59.0, -58.0, -57.0, -56.0, -55.0, -54.0, -53.0, -52.0],
                 dtype='float64', name='longitude'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1836-01-01 00:00:00', '1836-01-01 03:00:00',
                   '1836-01-01 06:00:00', '1836-01-01 09:00:00',
                   '1836-01-01 12:00:00', '1836-01-01 15:00:00',
                   '1836-01-01 18:00:00', '1836-01-01 21:00:00',
                   '1836-01-02 00:00:00', '1836-01-02 03:00:00',
                   ...
                   '2015-12-30 18:00:00', '2015-12-30 21:00:00',
                   '2015-12-31 00:00:00', '2015-12-31 03:00:00',
                   '2015-12-31 06:00:00', '2015-12-31 09:00:00',
                   '2015-12-31 12:00:00', '2015-12-31 15:00:00',
                   '2015-12-31 18:00:00', '2015-12-31 21:00:00'],
                  dtype='datetime64[ns]', name='time', length=525952, freq=None))

• Attributes: (25)

  Conventions :

  CF-1.2

  DODS_EXTRA.Unlimited_Dimension :

  time

  References :

  https://www.esrl.noaa.gov/psd/data/gridded/data.20thC_ReanV3.html

  assimilation_algorithm :

  Ensemble Kalman Filter with 4DIAU

  citation :

  Compo,G.P. <https://www.esrl.noaa.gov/psd/people/gilbert.p.compo>, J.S. Whitaker <https://www.esrl.noaa.gov/psd/people/jeffrey.s.whitaker> P.D. Sardeshmukh, N.Matsui, R.J. Allan, X. Yin, B.E. Gleason, R.S. Vose, G. Rutledge, P. Bessemoulin, S. Bronnimann, M. Brunet, R.I. Crouthamel, A.N. Grant, P.Y. Groisman, P.D. Jones, M. Kruk, A.C. Kruger, G.J. Marshal, M. Maugeri, H.Y. Mok, O. Nordli, T.F. Ross, R.M. Trio, X.L. Wang, S.D. Woodruff, and S.J. Worley, 2011: The Twentieth Century Reanalysis Project <http://onlinelibrary.wiley.com/doi/10.1002/qj.776/abstract>. Quarterly J. Roy. Meteorol. Soc./, 137, 1-28. DOI: 10.1002/qj.776.

  citation1 :

  Slivinski, L. C, G. P. Compo, J. S. Whitaker, P. D. Sardeshmukh, B. S. Giese, C. McColl, R. Allan, X. Yin, R. Vose, H. Titchner, J. Kennedy, L. J. Spencer, L. Ashcroft, S. Bronnimann, M. Brunet, D. Camuffo, R. Cornes, T. A. Cram, R. Crouthamel, F. Dominguez-Castro, J. E. Freeman, J. Gergis, E. Hawkins, P. D. Jones, S. Jourdain, A. Kaplan, H. Kubota, F. Le Blancq, T.-C. Lee, A. Lorrey, J. Luterbacher, M. Maugeri, C. J. Mock, G.W. K. Moore, R. Przybylak, C. Pudmenzky, C. Reason, V. C. Slonosky, C. Smith, B. Tinz, B. Trewin, M. A. Valente, X. L. Wang, C. Wilkinson, K. Wood, and P. Wyszynski: Towards a more reliable historical reanalysis: Improvements for version 3 of the Twentieth Century Reanalysis System. Quarterly Journal of the Royal Meteorological Society. 2019; 145: 2876- 2908. doi:10.1002/qj.3598.

  comments :

  Data are from NOAA/CIRES/DOE 20th Century Reanalysis version 3 (8x/day). It consists of most variables interpolated to pressure surfaces from model hybrid (sigma/pressure) surfaces.

  contact :

  esrl.psd.data@noaa.gov

  creation_date :

  2018-11-02T19:30:07Z

  creator_name :

  NOAA/ESRL PSD

  dataset_title :

  NOAA/CIRES/DOE 20th Century Reanalysis (V3)

  forcing :

  CO2, Sl, Vl, SST, Oz (CO2 as in Saha et al (2010), Oz is prescribed from CMIP5, SST is SODAsi.3, Sl is Total Solar Irradiance (TSI) Reconstruction based on NRLTSI2, Vl defined as in Crowley & Unterman 2013)

  forcing_note :

  Additional information on the external forcings uses in this reanalysis can be found in Slivinski et al. (2019) (citation1)

  history :

  created 2018/11 by Hoop (chunked, deflated non-packed netCDF4-Classic) esrl.psd.data@noaa.gov

  institution :

  NOAA ESRL Physical Sciences Division & CU/CIRES (325 Broadway R/PSD, Boulder, CO 80305)

  license :

  These data are available free of charge under the PSD data policy

  name_of_model_used :

  ensda_v451

  observations :

  International Surface Pressure Databank version 4.7

  platform :

  Model

  product :

  reanalysis

  source :

  20CRv3si 2018, Ensemble Kalman Filter, ocean (specified): SODAsi.3; atmosphere: NCEP GFS v14.0.1; sea ice (specified): HadISST2.3 (years prior to 1850 use climatology of 1860-1891); land: Noah 4 layer version 2

  spatial_resolution :

  1.0 degree

  standard_name_vocabulary :

  NetCDF Climate and Forecast (CF) Metadata Convention

  title :

  8x Daily NOAA/CIRES/DOE 20th Century Reanalysis V3

  version :

  3si

ds=cat.atmosphere['20_century_reanalysis_pressure_levels_large_area'].to_dask()
ds

<xarray.Dataset>
Dimensions:    (time: 525952, level: 17, latitude: 24, longitude: 45)
Coordinates:
  * latitude   (latitude) float32 40.0 41.0 42.0 43.0 ... 60.0 61.0 62.0 63.0
  * level      (level) float64 1.0 5.0 10.0 20.0 ... 700.0 800.0 900.0 1e+03
  * longitude  (longitude) float32 -96.0 -95.0 -94.0 -93.0 ... -54.0 -53.0 -52.0
  * time       (time) datetime64[ns] 1836-01-01 ... 2015-12-31T21:00:00
Data variables:
    air        (time, level, latitude, longitude) float32 dask.array<chunksize=(100, 1, 24, 45), meta=np.ndarray>
    hgt        (time, level, latitude, longitude) float32 dask.array<chunksize=(100, 1, 24, 45), meta=np.ndarray>
    omega      (time, level, latitude, longitude) float32 dask.array<chunksize=(100, 1, 24, 45), meta=np.ndarray>
    rhum       (time, level, latitude, longitude) float32 dask.array<chunksize=(100, 1, 24, 45), meta=np.ndarray>
Attributes: (12/25)
    Conventions:                     CF-1.2
    DODS_EXTRA.Unlimited_Dimension:  time
    References:                      https://www.esrl.noaa.gov/psd/data/gridd...
    assimilation_algorithm:          Ensemble Kalman Filter with 4DIAU
    citation:                        Compo,G.P. <https://www.esrl.noaa.gov/ps...
    citation1:                       Slivinski, L. C, G. P. Compo, J. S. Whit...
    ...                              ...
    product:                         reanalysis
    source:                          20CRv3si 2018, Ensemble Kalman Filter, o...
    spatial_resolution:              1.0 degree
    standard_name_vocabulary:        NetCDF Climate and Forecast (CF) Metadat...
    title:                           8x Daily NOAA/CIRES/DOE 20th Century Rea...
    version:                         3si

xarray.Dataset

• Dimensions:
  • time: 525952
  • level: 17
  • latitude: 24
  • longitude: 45
• Coordinates: (4)
  • latitude
    (latitude)
    float32
    40.0 41.0 42.0 ... 61.0 62.0 63.0

    actual_range :

    [90.0, -90.0]

    axis :

    Y

    coordinate_defines :

    point

    long_name :

    Latitude

    standard_name :

    latitude

    units :

    degrees_north

    array([40., 41., 42., 43., 44., 45., 46., 47., 48., 49., 50., 51., 52., 53.,
           54., 55., 56., 57., 58., 59., 60., 61., 62., 63.], dtype=float32)

  • level
    (level)
    float64
    1.0 5.0 10.0 ... 800.0 900.0 1e+03

    actual_range :

    [1000.0, 1.0]

    axis :

    Z

    coordinate_defines :

    point

    long_name :

    Level

    positive :

    down

    units :

    millibar

    array([   1.,    5.,   10.,   20.,   30.,   50.,   70.,  100.,  200.,  300.,
            400.,  500.,  600.,  700.,  800.,  900., 1000.])

  • longitude
    (longitude)
    float32
    -96.0 -95.0 -94.0 ... -53.0 -52.0

    array([-96., -95., -94., -93., -92., -91., -90., -89., -88., -87., -86., -85.,
           -84., -83., -82., -81., -80., -79., -78., -77., -76., -75., -74., -73.,
           -72., -71., -70., -69., -68., -67., -66., -65., -64., -63., -62., -61.,
           -60., -59., -58., -57., -56., -55., -54., -53., -52.], dtype=float32)

  • time
    (time)
    datetime64[ns]
    1836-01-01 ... 2015-12-31T21:00:00

    _ChunkSizes :

    1

    axis :

    T

    coordinate_defines :

    point

    delta_t :

    0000-00-00 03:00:00

    long_name :

    Time

    standard_name :

    time

    array(['1836-01-01T00:00:00.000000000', '1836-01-01T03:00:00.000000000',
           '1836-01-01T06:00:00.000000000', ..., '2015-12-31T15:00:00.000000000',
           '2015-12-31T18:00:00.000000000', '2015-12-31T21:00:00.000000000'],
          dtype='datetime64[ns]')

• Data variables: (4)
  • air
    (time, level, latitude, longitude)
    float32
    dask.array<chunksize=(100, 1, 24, 45), meta=np.ndarray>

    GRIB_name :

    TMP

    GridType :

    Cylindrical Equidistant Projection Grid

    _ChunkSizes :

    [1, 1, 181, 360]

    actual_range :

    [188.0, 321.8634338378906]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Pressure Levels

    long_name :

    3-hourly Air Temperature on Pressure Levels

    parent_stat :

    Other

    standard_name :

    air_temperature

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    degK

    valid_range :

    [100.0, 500.0]

    var_desc :

    Air temperature

    Array Chunk Bytes 35.97 GiB 421.88 kiB Shape (525952, 17, 24, 45) (100, 1, 24, 45) Dask graph 89420 chunks in 2 graph layers Data type float32 numpy.ndarray

  • hgt
    (time, level, latitude, longitude)
    float32
    dask.array<chunksize=(100, 1, 24, 45), meta=np.ndarray>

    GRIB_name :

    HGT

    GridType :

    Cylindrical Equidistant Projection Grid

    _ChunkSizes :

    [1, 1, 181, 360]

    actual_range :

    [-283.2093200683594, 50456.0703125]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Pressure Levels

    long_name :

    3-hourly Geopotential Height on Pressure Levels

    parent_stat :

    Other

    standard_name :

    geopotential_height

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    m

    valid_range :

    [-1500.0, 60000.0]

    var_desc :

    Geopotential height

    Array Chunk Bytes 35.97 GiB 421.88 kiB Shape (525952, 17, 24, 45) (100, 1, 24, 45) Dask graph 89420 chunks in 2 graph layers Data type float32 numpy.ndarray

  • omega
    (time, level, latitude, longitude)
    float32
    dask.array<chunksize=(100, 1, 24, 45), meta=np.ndarray>

    GRIB_name :

    VVEL

    GridType :

    Cylindrical Equidistant Projection Grid

    _ChunkSizes :

    [1, 1, 181, 360]

    actual_range :

    [-1.5944035053253174, 2.981666326522827]

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Pressure Levels

    long_name :

    3-hourly Omega on Pressure Levels

    parent_stat :

    Other

    standard_name :

    vertical_air_velocity_expressed_as_tendency_of_pressure

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    Pascal/s

    valid_range :

    [-15.0, 15.0]

    var_desc :

    Omega (dp/dt)

    Array Chunk Bytes 35.97 GiB 421.88 kiB Shape (525952, 17, 24, 45) (100, 1, 24, 45) Dask graph 89420 chunks in 2 graph layers Data type float32 numpy.ndarray

  • rhum
    (time, level, latitude, longitude)
    float32
    dask.array<chunksize=(100, 1, 24, 45), meta=np.ndarray>

    GRIB_name :

    RH

    GridType :

    Cylindrical Equidistant Projection Grid

    _ChunkSizes :

    [1, 1, 181, 360]

    actual_range :

    [0.0, 100.0]

    comment :

    Calculated from the ensemble mean and not averaged from the relative humidity of individual ensemble members. The values are similar but not exact

    dataset :

    NOAA/CIRES/DOE 20th Century Reanalysis version 3si

    datum :

    wgs84

    level_desc :

    Pressure Levels

    long_name :

    3-hourly Relative Humidity on Pressure Levels

    parent_stat :

    Other

    standard_name :

    relative_humidity

    statistic :

    Ensemble Mean

    statistic_method :

    Ensemble mean is calculated by averaging over all 80 ensemble members at each time step

    units :

    %

    valid_range :

    [0.0, 100.0]

    var_desc :

    Relative Humidity

    Array Chunk Bytes 35.97 GiB 421.88 kiB Shape (525952, 17, 24, 45) (100, 1, 24, 45) Dask graph 89420 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (4)
  • latitude
    PandasIndex

    PandasIndex(Float64Index([40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0,
                  51.0, 52.0, 53.0, 54.0, 55.0, 56.0, 57.0, 58.0, 59.0, 60.0, 61.0,
                  62.0, 63.0],
                 dtype='float64', name='latitude'))

  • level
    PandasIndex

    PandasIndex(Float64Index([   1.0,    5.0,   10.0,   20.0,   30.0,   50.0,   70.0,  100.0,
                   200.0,  300.0,  400.0,  500.0,  600.0,  700.0,  800.0,  900.0,
                  1000.0],
                 dtype='float64', name='level'))

  • longitude
    PandasIndex

    PandasIndex(Float64Index([-96.0, -95.0, -94.0, -93.0, -92.0, -91.0, -90.0, -89.0, -88.0,
                  -87.0, -86.0, -85.0, -84.0, -83.0, -82.0, -81.0, -80.0, -79.0,
                  -78.0, -77.0, -76.0, -75.0, -74.0, -73.0, -72.0, -71.0, -70.0,
                  -69.0, -68.0, -67.0, -66.0, -65.0, -64.0, -63.0, -62.0, -61.0,
                  -60.0, -59.0, -58.0, -57.0, -56.0, -55.0, -54.0, -53.0, -52.0],
                 dtype='float64', name='longitude'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1836-01-01 00:00:00', '1836-01-01 03:00:00',
                   '1836-01-01 06:00:00', '1836-01-01 09:00:00',
                   '1836-01-01 12:00:00', '1836-01-01 15:00:00',
                   '1836-01-01 18:00:00', '1836-01-01 21:00:00',
                   '1836-01-02 00:00:00', '1836-01-02 03:00:00',
                   ...
                   '2015-12-30 18:00:00', '2015-12-30 21:00:00',
                   '2015-12-31 00:00:00', '2015-12-31 03:00:00',
                   '2015-12-31 06:00:00', '2015-12-31 09:00:00',
                   '2015-12-31 12:00:00', '2015-12-31 15:00:00',
                   '2015-12-31 18:00:00', '2015-12-31 21:00:00'],
                  dtype='datetime64[ns]', name='time', length=525952, freq=None))

• Attributes: (25)

  Conventions :

  CF-1.2

  DODS_EXTRA.Unlimited_Dimension :

  time

  References :

  https://www.esrl.noaa.gov/psd/data/gridded/data.20thC_ReanV3.html

  assimilation_algorithm :

  Ensemble Kalman Filter with 4DIAU

  citation :

  Compo,G.P. <https://www.esrl.noaa.gov/psd/people/gilbert.p.compo>, J.S. Whitaker <https://www.esrl.noaa.gov/psd/people/jeffrey.s.whitaker> P.D. Sardeshmukh, N.Matsui, R.J. Allan, X. Yin, B.E. Gleason, R.S. Vose, G. Rutledge, P. Bessemoulin, S. Bronnimann, M. Brunet, R.I. Crouthamel, A.N. Grant, P.Y. Groisman, P.D. Jones, M. Kruk, A.C. Kruger, G.J. Marshal, M. Maugeri, H.Y. Mok, O. Nordli, T.F. Ross, R.M. Trio, X.L. Wang, S.D. Woodruff, and S.J. Worley, 2011: The Twentieth Century Reanalysis Project <http://onlinelibrary.wiley.com/doi/10.1002/qj.776/abstract>. Quarterly J. Roy. Meteorol. Soc./, 137, 1-28. DOI: 10.1002/qj.776.

  citation1 :

  Slivinski, L. C, G. P. Compo, J. S. Whitaker, P. D. Sardeshmukh, B. S. Giese, C. McColl, R. Allan, X. Yin, R. Vose, H. Titchner, J. Kennedy, L. J. Spencer, L. Ashcroft, S. Bronnimann, M. Brunet, D. Camuffo, R. Cornes, T. A. Cram, R. Crouthamel, F. Dominguez-Castro, J. E. Freeman, J. Gergis, E. Hawkins, P. D. Jones, S. Jourdain, A. Kaplan, H. Kubota, F. Le Blancq, T.-C. Lee, A. Lorrey, J. Luterbacher, M. Maugeri, C. J. Mock, G.W. K. Moore, R. Przybylak, C. Pudmenzky, C. Reason, V. C. Slonosky, C. Smith, B. Tinz, B. Trewin, M. A. Valente, X. L. Wang, C. Wilkinson, K. Wood, and P. Wyszynski: Towards a more reliable historical reanalysis: Improvements for version 3 of the Twentieth Century Reanalysis System. Quarterly Journal of the Royal Meteorological Society. 2019; 145: 2876- 2908. doi:10.1002/qj.3598.

  comments :

  Data are from NOAA/CIRES/DOE 20th Century Reanalysis version 3 (8x/day). It consists of most variables interpolated to pressure surfaces from model hybrid (sigma/pressure) surfaces.

  contact :

  esrl.psd.data@noaa.gov

  creation_date :

  2018-11-02T19:30:07Z

  creator_name :

  NOAA/ESRL PSD

  dataset_title :

  NOAA/CIRES/DOE 20th Century Reanalysis (V3)

  forcing :

  CO2, Sl, Vl, SST, Oz (CO2 as in Saha et al (2010), Oz is prescribed from CMIP5, SST is SODAsi.3, Sl is Total Solar Irradiance (TSI) Reconstruction based on NRLTSI2, Vl defined as in Crowley & Unterman 2013)

  forcing_note :

  Additional information on the external forcings uses in this reanalysis can be found in Slivinski et al. (2019) (citation1)

  history :

  created 2018/11 by Hoop (chunked, deflated non-packed netCDF4-Classic) esrl.psd.data@noaa.gov

  institution :

  NOAA ESRL Physical Sciences Division & CU/CIRES (325 Broadway R/PSD, Boulder, CO 80305)

  license :

  These data are available free of charge under the PSD data policy

  name_of_model_used :

  ensda_v451

  observations :

  International Surface Pressure Databank version 4.7

  platform :

  Model

  product :

  reanalysis

  source :

  20CRv3si 2018, Ensemble Kalman Filter, ocean (specified): SODAsi.3; atmosphere: NCEP GFS v14.0.1; sea ice (specified): HadISST2.3 (years prior to 1850 use climatology of 1860-1891); land: Noah 4 layer version 2

  spatial_resolution :

  1.0 degree

  standard_name_vocabulary :

  NetCDF Climate and Forecast (CF) Metadata Convention

  title :

  8x Daily NOAA/CIRES/DOE 20th Century Reanalysis V3

  version :

  3si

ds=cat.atmosphere['terraclimate'].to_dask()
ds

<xarray.Dataset>
Dimensions:                 (time: 744, lat: 4320, lon: 8640, crs: 1)
Coordinates:
  * crs                     (crs) int16 3
  * lat                     (lat) float64 89.98 89.94 89.9 ... -89.94 -89.98
  * lon                     (lon) float64 -180.0 -179.9 -179.9 ... 179.9 180.0
  * time                    (time) datetime64[ns] 1958-01-01 ... 2019-12-01
Data variables: (12/18)
    aet                     (time, lat, lon) float32 dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>
    def                     (time, lat, lon) float32 dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>
    pdsi                    (time, lat, lon) float32 dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>
    pet                     (time, lat, lon) float32 dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>
    ppt                     (time, lat, lon) float32 dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>
    ppt_station_influence   (time, lat, lon) float32 dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>
    ...                      ...
    tmin                    (time, lat, lon) float32 dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>
    tmin_station_influence  (time, lat, lon) float32 dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>
    vap                     (time, lat, lon) float32 dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>
    vap_station_influence   (time, lat, lon) float32 dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>
    vpd                     (time, lat, lon) float32 dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>
    ws                      (time, lat, lon) float32 dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

xarray.Dataset

• Dimensions:
  • time: 744
  • lat: 4320
  • lon: 8640
  • crs: 1
• Coordinates: (4)
  • crs
    (crs)
    int16
    3

    grid_mapping_name :

    latitude_longitude

    inverse_flattening :

    [298.257223563]

    long_name :

    crs

    longitude_of_prime_meridian :

    [0.0]

    semi_major_axis :

    [6378137.0]

    array([3], dtype=int16)

  • lat
    (lat)
    float64
    89.98 89.94 89.9 ... -89.94 -89.98

    axis :

    Y

    description :

    Latitude of the center of the grid cell

    long_name :

    latitude

    standard_name :

    latitude

    units :

    degrees_north

    array([ 89.979167,  89.9375  ,  89.895833, ..., -89.895833, -89.9375  ,
           -89.979167])

  • lon
    (lon)
    float64
    -180.0 -179.9 ... 179.9 180.0

    axis :

    X

    description :

    Longitude of the center of the grid cell

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    array([-179.979167, -179.9375  , -179.895833, ...,  179.895833,  179.9375  ,
            179.979167])

  • time
    (time)
    datetime64[ns]
    1958-01-01 ... 2019-12-01

    axis :

    T

    description :

    days since 1900-01-01

    long_name :

    time

    standard_name :

    time

    array(['1958-01-01T00:00:00.000000000', '1958-02-01T00:00:00.000000000',
           '1958-03-01T00:00:00.000000000', ..., '2019-10-01T00:00:00.000000000',
           '2019-11-01T00:00:00.000000000', '2019-12-01T00:00:00.000000000'],
          dtype='datetime64[ns]')

• Data variables: (18)
  • aet
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    Actual Evapotranspiration

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    water_evaporation_amount

    standard_name :

    water_evaporation_amount

    units :

    mm

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • def
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    Climatic Water Deficit

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    water_potential_evaporation_amount_minus_water_evaporation_amount

    standard_name :

    water_potential_evaporation_amount_minus_water_evaporation_amount

    units :

    mm

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • pdsi
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    Palmer Drought Severity Index

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    palmer_drought_severity_index

    standard_name :

    palmer_drought_severity_index

    units :

    unitless

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • pet
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    Reference Evapotranspiration

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    water_potential_evaporation_amount

    standard_name :

    water_potential_evaporation_amount

    units :

    mm

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • ppt
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    Accumulated Precipitation

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    precipitation_amount

    standard_name :

    precipitation_amount

    units :

    mm

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • ppt_station_influence
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    A number between 0 and 8, indicating the number of stations contributing to each grid value directly from CRU and interpolated to the TerraClimate grid. When this value is greater than 1, CRU data is used for the anomalies in the method. When this value is 0, JRA-55/ERA-20C/ERA-INTERIM is used for anomalies in the method.

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    station_influence

    standard_name :

    station_influence

    units :

    none

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • q
    (time, lat, lon)
    float64
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    Runoff

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    runoff_amount

    standard_name :

    runoff_amount

    units :

    mm

    Array Chunk Bytes 206.90 GiB 189.84 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float64 numpy.ndarray

  • soil
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    Soil Moisture at End of Month

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    soil_moisture_content

    standard_name :

    soil_moisture_content

    units :

    mm

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • srad
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    Downward Shortwave Radiation Flux at the Surface

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    downwelling_shortwave_flux_in_air

    standard_name :

    downwelling_shortwave_flux_in_air

    units :

    W/m2

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • swe
    (time, lat, lon)
    float64
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    Snow Water Equivalent at End of Month

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    liquid_water_content_of_surface_snow

    standard_name :

    liquid_water_content_of_surface_snow

    units :

    mm

    Array Chunk Bytes 206.90 GiB 189.84 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float64 numpy.ndarray

  • tmax
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    Maximum 2-m Temperature

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    air_temperature

    standard_name :

    air_temperature

    units :

    deg C

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tmax_station_influence
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    A number between 0 and 8, indicating the number of stations contributing to each grid value directly from CRU and interpolated to the TerraClimate grid. When this value is greater than 1, CRU data is used for the anomalies in the method. When this value is 0, JRA-55/ERA-20C/ERA-INTERIM is used for anomalies in the method.

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    station_influence

    standard_name :

    station_influence

    units :

    none

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tmin
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    Minimum 2-m Temperature

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    air_temperature

    standard_name :

    air_temperature

    units :

    deg C

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tmin_station_influence
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    A number between 0 and 8, indicating the number of stations contributing to each grid value directly from CRU and interpolated to the TerraClimate grid. When this value is greater than 1, CRU data is used for the anomalies in the method. When this value is 0, JRA-55/ERA-20C/ERA-INTERIM is used for anomalies in the method.

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    station_influence

    standard_name :

    station_influence

    units :

    none

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • vap
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    2-m Vapor Pressure

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    water_vapor_partial_pressure_in_air

    standard_name :

    water_vapor_partial_pressure_in_air

    units :

    kPa

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • vap_station_influence
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    A number between 0 and 8, indicating the number of stations contributing to each grid value directly from CRU and interpolated to the TerraClimate grid. When this value is greater than 1, CRU data is used for the anomalies in the method. When this value is 0, JRA-55/ERA-20C/ERA-INTERIM is used for anomalies in the method.

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    station_influence

    standard_name :

    station_influence

    units :

    none

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • vpd
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    Mean Vapor Pressure Deficit

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    vapor_pressure_deficit

    standard_name :

    vapor_pressure_deficit

    units :

    kPa

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

  • ws
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1440, 1440), meta=np.ndarray>

    coordinate_system :

    WGS84,EPSG:4326

    description :

    10-m Wind Speed

    dimensions :

    lon lat time

    grid_mapping :

    crs

    long_name :

    wind_speed

    standard_name :

    wind_speed

    units :

    m/s

    Array Chunk Bytes 103.45 GiB 94.92 MiB Shape (744, 4320, 8640) (12, 1440, 1440) Dask graph 1116 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (4)
  • crs
    PandasIndex

    PandasIndex(Int64Index([3], dtype='int64', name='crs'))

  • lat
    PandasIndex

    PandasIndex(Float64Index([ 89.97916666666667,            89.9375,  89.89583333333334,
                   89.85416666666667,            89.8125,  89.77083333333334,
                   89.72916666666667,            89.6875,  89.64583333333334,
                   89.60416666666667,
                  ...
                  -89.60416666666664, -89.64583333333331, -89.68749999999997,
                  -89.72916666666664, -89.77083333333331, -89.81249999999997,
                  -89.85416666666664, -89.89583333333331, -89.93749999999997,
                  -89.97916666666664],
                 dtype='float64', name='lat', length=4320))

  • lon
    PandasIndex

    PandasIndex(Float64Index([-179.97916666666666,           -179.9375, -179.89583333333331,
                  -179.85416666666666,           -179.8125, -179.77083333333331,
                  -179.72916666666666,           -179.6875, -179.64583333333331,
                  -179.60416666666666,
                  ...
                   179.60416666666666,  179.64583333333331,            179.6875,
                   179.72916666666666,  179.77083333333331,            179.8125,
                   179.85416666666666,  179.89583333333331,            179.9375,
                   179.97916666666666],
                 dtype='float64', name='lon', length=8640))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1958-01-01', '1958-02-01', '1958-03-01', '1958-04-01',
                   '1958-05-01', '1958-06-01', '1958-07-01', '1958-08-01',
                   '1958-09-01', '1958-10-01',
                   ...
                   '2019-03-01', '2019-04-01', '2019-05-01', '2019-06-01',
                   '2019-07-01', '2019-08-01', '2019-09-01', '2019-10-01',
                   '2019-11-01', '2019-12-01'],
                  dtype='datetime64[ns]', name='time', length=744, freq=None))

• Attributes: (0)

# A new dataset with all rcps is being created and will replace this one
ds=cat.climate_change['rcp45_day_NAM_22i_raw_zarr'].to_dask()
ds

<xarray.Dataset>
Dimensions:    (lat: 258, lon: 600, member_id: 3, time: 34698, bnds: 2)
Coordinates:
  * lat        (lat) float64 12.12 12.38 12.62 12.88 ... 75.62 75.88 76.12 76.38
  * lon        (lon) float64 -171.9 -171.6 -171.4 ... -22.62 -22.38 -22.12
  * member_id  (member_id) <U20 'CanESM2.CRCM5-OUR' ... 'GFDL-ESM2M.CRCM5-OUR'
  * time       (time) datetime64[ns] 2006-01-01T12:00:00 ... 2100-12-31T12:00:00
    time_bnds  (time, bnds) datetime64[ns] dask.array<chunksize=(17349, 2), meta=np.ndarray>
Dimensions without coordinates: bnds
Data variables: (12/15)
    hurs       (member_id, time, lat, lon) float32 dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>
    huss       (member_id, time, lat, lon) float32 dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>
    pr         (member_id, time, lat, lon) float32 dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>
    prec       (member_id, time, lat, lon) float32 dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>
    ps         (member_id, time, lat, lon) float32 dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>
    rsds       (member_id, time, lat, lon) float32 dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>
    ...         ...
    tasmin     (member_id, time, lat, lon) float32 dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>
    temp       (member_id, time, lat, lon) float32 dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>
    tmax       (member_id, time, lat, lon) float32 dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>
    tmin       (member_id, time, lat, lon) float32 dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>
    uas        (member_id, time, lat, lon) float32 dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>
    vas        (member_id, time, lat, lon) float32 dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>
Attributes: (12/23)
    CORDEX_domain:                  NAM-22
    contact:                        {"GFDL-ESM2M.CRCM5-OUR": "biner.sebastien...
    creation_date:                  {"GFDL-ESM2M.CRCM5-OUR": "2019-02-12 15:2...
    driving_experiment:             {"GFDL-ESM2M.CRCM5-OUR": "GFDL-ESM2M,rcp4...
    driving_experiment_name:        rcp45
    driving_model_ensemble_member:  {"GFDL-ESM2M.CRCM5-OUR": "r1i1p1", "CanES...
    ...                             ...
    references:                     {"GFDL-ESM2M.CRCM5-OUR": "http://www.oura...
    title:                          {"GFDL-ESM2M.CRCM5-OUR": "NA-CORDEX Raw N...
    tracking_id:                    {"GFDL-ESM2M.CRCM5-OUR": "5139ec82-c55f-4...
    version:                        {"GFDL-ESM2M.CRCM5-OUR": "1.1", "CanESM2....
    zarr-dataset-reference:         For dataset documentation, see DOI https:...
    zarr-version:                   1.0

xarray.Dataset

• Dimensions:
  • lat: 258
  • lon: 600
  • member_id: 3
  • time: 34698
  • bnds: 2
• Coordinates: (5)
  • lat
    (lat)
    float64
    12.12 12.38 12.62 ... 76.12 76.38

    long_name :

    latitude

    standard_name :

    latitude

    units :

    degrees_north

    array([12.125, 12.375, 12.625, ..., 75.875, 76.125, 76.375])

  • lon
    (lon)
    float64
    -171.9 -171.6 ... -22.38 -22.12

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    array([-171.875, -171.625, -171.375, ...,  -22.625,  -22.375,  -22.125])

  • member_id
    (member_id)
    <U20
    'CanESM2.CRCM5-OUR' ... 'GFDL-ES...

    array(['CanESM2.CRCM5-OUR', 'CanESM2.CanRCM4', 'GFDL-ESM2M.CRCM5-OUR'],
          dtype='<U20')

  • time
    (time)
    datetime64[ns]
    2006-01-01T12:00:00 ... 2100-12-...

    array(['2006-01-01T12:00:00.000000000', '2006-01-02T12:00:00.000000000',
           '2006-01-03T12:00:00.000000000', ..., '2100-12-29T12:00:00.000000000',
           '2100-12-30T12:00:00.000000000', '2100-12-31T12:00:00.000000000'],
          dtype='datetime64[ns]')

  • time_bnds
    (time, bnds)
    datetime64[ns]
    dask.array<chunksize=(17349, 2), meta=np.ndarray>

    Array Chunk Bytes 542.16 kiB 271.08 kiB Shape (34698, 2) (17349, 2) Dask graph 2 chunks in 72 graph layers Data type datetime64[ns] numpy.ndarray

• Data variables: (15)
  • hurs
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time: mean

    long_name :

    Near-Surface Relative Humidity

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    relative_humidity

    units :

    %

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 3 graph layers Data type float32 numpy.ndarray

  • huss
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time: mean

    long_name :

    Near-Surface Specific Humidity

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    specific_humidity

    units :

    kg kg-1

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 3 graph layers Data type float32 numpy.ndarray

  • pr
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time: mean

    long_name :

    Precipitation

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    precipitation_flux

    units :

    kg m-2 s-1

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 3 graph layers Data type float32 numpy.ndarray

  • prec
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time: mean

    long_name :

    Precipitation

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    lwe_precipitation_rate

    units :

    mm/day

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 3 graph layers Data type float32 numpy.ndarray

  • ps
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time: mean

    long_name :

    Surface Air Pressure

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    surface_air_pressure

    units :

    Pa

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 3 graph layers Data type float32 numpy.ndarray

  • rsds
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time:mean (interval:600 seconds) time: mean

    long_name :

    Surface Downwelling Shortwave Radiation

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    surface_downwelling_shortwave_flux_in_air

    units :

    W m-2

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 16 graph layers Data type float32 numpy.ndarray

  • sfcWind
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time: mean

    long_name :

    Near-Surface Wind Speed

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    wind_speed

    units :

    m s-1

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 16 graph layers Data type float32 numpy.ndarray

  • tas
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time: mean

    long_name :

    Near-Surface Air Temperature

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    air_temperature

    units :

    K

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 3 graph layers Data type float32 numpy.ndarray

  • tasmax
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time: maximum

    long_name :

    Daily Maximum Near-Surface Air Temperature

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    air_temperature

    units :

    K

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 3 graph layers Data type float32 numpy.ndarray

  • tasmin
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time: minimum

    long_name :

    Daily Minimum Near-Surface Air Temperature

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    air_temperature

    units :

    K

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 3 graph layers Data type float32 numpy.ndarray

  • temp
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time: mean

    long_name :

    Near-Surface Air Temperature

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    air_temperature

    units :

    degC

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 3 graph layers Data type float32 numpy.ndarray

  • tmax
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time: maximum

    long_name :

    Daily Maximum Near-Surface Air Temperature

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    air_temperature

    units :

    degC

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 3 graph layers Data type float32 numpy.ndarray

  • tmin
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time: minimum

    long_name :

    Daily Minimum Near-Surface Air Temperature

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    air_temperature

    units :

    degC

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 3 graph layers Data type float32 numpy.ndarray

  • uas
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time:mean (interval:600 seconds)

    long_name :

    Eastward Near-Surface Wind

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    eastward_wind

    units :

    m s-1

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 16 graph layers Data type float32 numpy.ndarray

  • vas
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(3, 1000, 65, 120), meta=np.ndarray>

    cell_methods :

    time:mean (interval:600 seconds)

    long_name :

    Northward Near-Surface Wind

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    northward_wind

    units :

    m s-1

    Array Chunk Bytes 60.03 GiB 89.26 MiB Shape (3, 34698, 258, 600) (3, 1000, 65, 120) Dask graph 700 chunks in 16 graph layers Data type float32 numpy.ndarray

• Indexes: (4)
  • lat
    PandasIndex

    PandasIndex(Float64Index([12.125, 12.375, 12.625, 12.875, 13.125, 13.375, 13.625, 13.875,
                  14.125, 14.375,
                  ...
                  74.125, 74.375, 74.625, 74.875, 75.125, 75.375, 75.625, 75.875,
                  76.125, 76.375],
                 dtype='float64', name='lat', length=258))

  • lon
    PandasIndex

    PandasIndex(Float64Index([-171.875, -171.625, -171.375, -171.125, -170.875, -170.625,
                  -170.375, -170.125, -169.875, -169.625,
                  ...
                   -24.375,  -24.125,  -23.875,  -23.625,  -23.375,  -23.125,
                   -22.875,  -22.625,  -22.375,  -22.125],
                 dtype='float64', name='lon', length=600))

  • member_id
    PandasIndex

    PandasIndex(Index(['CanESM2.CRCM5-OUR', 'CanESM2.CanRCM4', 'GFDL-ESM2M.CRCM5-OUR'], dtype='object', name='member_id'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2006-01-01 12:00:00', '2006-01-02 12:00:00',
                   '2006-01-03 12:00:00', '2006-01-04 12:00:00',
                   '2006-01-05 12:00:00', '2006-01-06 12:00:00',
                   '2006-01-07 12:00:00', '2006-01-08 12:00:00',
                   '2006-01-09 12:00:00', '2006-01-10 12:00:00',
                   ...
                   '2100-12-22 12:00:00', '2100-12-23 12:00:00',
                   '2100-12-24 12:00:00', '2100-12-25 12:00:00',
                   '2100-12-26 12:00:00', '2100-12-27 12:00:00',
                   '2100-12-28 12:00:00', '2100-12-29 12:00:00',
                   '2100-12-30 12:00:00', '2100-12-31 12:00:00'],
                  dtype='datetime64[ns]', name='time', length=34698, freq=None))

• Attributes: (23)

  CORDEX_domain :

  NAM-22

  contact :

  {"GFDL-ESM2M.CRCM5-OUR": "biner.sebastien@ouranos.ca", "CanESM2.CRCM5-OUR": "paquin.dominique@ouranos.ca", "CanESM2.CanRCM4": "cccma_info@ec.gc.ca"}

  creation_date :

  {"GFDL-ESM2M.CRCM5-OUR": "2019-02-12 15:21", "CanESM2.CRCM5-OUR": "2019-01-11 16:10", "CanESM2.CanRCM4": "2012-12-03 19:35:57"}

  driving_experiment :

  {"GFDL-ESM2M.CRCM5-OUR": "GFDL-ESM2M,rcp45,r1i1p1", "CanESM2.CRCM5-OUR": "CCCma-CanESM2,rcp45.r1i1p1", "CanESM2.CanRCM4": "CCCma-CanESM2, rcp45, r1i1p1"}

  driving_experiment_name :

  rcp45

  driving_model_ensemble_member :

  {"GFDL-ESM2M.CRCM5-OUR": "r1i1p1", "CanESM2.CRCM5-OUR": "r1i1p1", "CanESM2.CanRCM4": "r1i1p1"}

  driving_model_id :

  {"GFDL-ESM2M.CRCM5-OUR": "NOAA-GFDL-GFDL-ESM2M", "CanESM2.CRCM5-OUR": "CCCma-CanESM2", "CanESM2.CanRCM4": "CCCma-CanESM2"}

  experiment_id :

  rcp45

  frequency :

  day

  id :

  doi:10.5065/D6SJ1JCH

  institute_id :

  {"GFDL-ESM2M.CRCM5-OUR": "OURANOS", "CanESM2.CRCM5-OUR": "OURANOS", "CanESM2.CanRCM4": "CCCma"}

  institution :

  {"GFDL-ESM2M.CRCM5-OUR": "Ouranos", "CanESM2.CRCM5-OUR": "Ouranos", "CanESM2.CanRCM4": "CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, Canada)"}

  model_id :

  {"GFDL-ESM2M.CRCM5-OUR": "OURANOS-CRCM5", "CanESM2.CRCM5-OUR": "OURANOS-CRCM5", "CanESM2.CanRCM4": "CCCma-CanRCM4"}

  original_calendar :

  {"GFDL-ESM2M.CRCM5-OUR": "365_day", "CanESM2.CRCM5-OUR": "365_day", "CanESM2.CanRCM4": "365_day"}

  product :

  output

  project_id :

  CORDEX

  rcm_version_id :

  {"GFDL-ESM2M.CRCM5-OUR": "v1", "CanESM2.CRCM5-OUR": "v1", "CanESM2.CanRCM4": "r2"}

  references :

  {"GFDL-ESM2M.CRCM5-OUR": "http://www.ouranos.ca", "CanESM2.CRCM5-OUR": "http://www.ouranos.ca", "CanESM2.CanRCM4": "http://www.cccma.ec.gc.ca/models"}

  title :

  {"GFDL-ESM2M.CRCM5-OUR": "NA-CORDEX Raw NAM-22i CRCM5-OUR GFDL-ESM2M RCP4.5 Daily Mean 2m Relative Humidity", "CanESM2.CRCM5-OUR": "NA-CORDEX Raw NAM-22i CRCM5-OUR CanESM2 RCP4.5 Daily Mean 2m Relative Humidity", "CanESM2.CanRCM4": "NA-CORDEX Raw NAM-22i CanRCM4 CanESM2 RCP4.5 Daily Mean 2m Relative Humidity"}

  tracking_id :

  {"GFDL-ESM2M.CRCM5-OUR": "5139ec82-c55f-4ecd-be73-81c51a1fe41f", "CanESM2.CRCM5-OUR": "b32afc1d-27a9-4cc4-8f39-4ec29f865a1d", "CanESM2.CanRCM4": "f52a798d-b519-4728-9801-6e812cccc7ea"}

  version :

  {"GFDL-ESM2M.CRCM5-OUR": "1.1", "CanESM2.CRCM5-OUR": "1.4", "CanESM2.CanRCM4": "2.3"}

  zarr-dataset-reference :

  For dataset documentation, see DOI https://doi.org/10.5065/D6SJ1JCH

  zarr-version :

  1.0

# Sample from melcc hydrometric data. Needs to be completed and add data from other providers as well.
ds=cat.hydrology['melcc'].to_dask()
ds

<xarray.Dataset>
Dimensions:                 (basin_id: 470, time: 41007)
Coordinates: (12/16)
    _last_update_timestamp  (basin_id) datetime64[ns] dask.array<chunksize=(470,), meta=np.ndarray>
    aggregation             (basin_id) <U1 dask.array<chunksize=(470,), meta=np.ndarray>
  * basin_id                (basin_id) <U6 '010101' '010801' ... '135201'
    data_type               (basin_id) <U1 dask.array<chunksize=(470,), meta=np.ndarray>
    drainage_area           (basin_id) float32 dask.array<chunksize=(470,), meta=np.ndarray>
    end_date                (basin_id) datetime64[ns] dask.array<chunksize=(470,), meta=np.ndarray>
    ...                      ...
    regulated               (basin_id) <U1 dask.array<chunksize=(470,), meta=np.ndarray>
    source                  (basin_id) <U1 dask.array<chunksize=(470,), meta=np.ndarray>
    start_date              (basin_id) datetime64[ns] dask.array<chunksize=(470,), meta=np.ndarray>
  * time                    (time) datetime64[ns] 1910-01-01 ... 2022-04-09
    timestep                (basin_id) <U1 dask.array<chunksize=(470,), meta=np.ndarray>
    units                   (basin_id) <U1 dask.array<chunksize=(470,), meta=np.ndarray>
Data variables:
    flag                    (time, basin_id) <U1 dask.array<chunksize=(2563, 59), meta=np.ndarray>
    value                   (time, basin_id) float32 dask.array<chunksize=(5126, 59), meta=np.ndarray>

xarray.Dataset

• Dimensions:
  • basin_id: 470
  • time: 41007
• Coordinates: (16)
  • _last_update_timestamp
    (basin_id)
    datetime64[ns]
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 3.67 kiB 3.67 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type datetime64[ns] numpy.ndarray

  • aggregation
    (basin_id)
    <U1
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 1.84 kiB 1.84 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type

  • basin_id
    (basin_id)
    <U6
    '010101' '010801' ... '135201'

    array(['010101', '010801', '010802', ..., '120201', '132901', '135201'],
          dtype='<U6')

  • data_type
    (basin_id)
    <U1
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 1.84 kiB 1.84 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type

  • drainage_area
    (basin_id)
    float32
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 1.84 kiB 1.84 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • end_date
    (basin_id)
    datetime64[ns]
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 3.67 kiB 3.67 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type datetime64[ns] numpy.ndarray

  • latitude
    (basin_id)
    float32
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 1.84 kiB 1.84 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • longitude
    (basin_id)
    float32
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 1.84 kiB 1.84 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • name
    (basin_id)
    <U1
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 1.84 kiB 1.84 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type

  • province
    (basin_id)
    <U1
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 1.84 kiB 1.84 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type

  • regulated
    (basin_id)
    <U1
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 1.84 kiB 1.84 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type

  • source
    (basin_id)
    <U1
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 1.84 kiB 1.84 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type

  • start_date
    (basin_id)
    datetime64[ns]
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 3.67 kiB 3.67 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type datetime64[ns] numpy.ndarray

  • time
    (time)
    datetime64[ns]
    1910-01-01 ... 2022-04-09

    array(['1910-01-01T00:00:00.000000000', '1910-01-02T00:00:00.000000000',
           '1910-01-03T00:00:00.000000000', ..., '2022-04-07T00:00:00.000000000',
           '2022-04-08T00:00:00.000000000', '2022-04-09T00:00:00.000000000'],
          dtype='datetime64[ns]')

  • timestep
    (basin_id)
    <U1
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 1.84 kiB 1.84 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type

  • units
    (basin_id)
    <U1
    dask.array<chunksize=(470,), meta=np.ndarray>

    Array Chunk Bytes 1.84 kiB 1.84 kiB Shape (470,) (470,) Dask graph 1 chunks in 2 graph layers Data type

• Data variables: (2)
  • flag
    (time, basin_id)
    <U1
    dask.array<chunksize=(2563, 59), meta=np.ndarray>

    Array Chunk Bytes 73.52 MiB 590.69 kiB Shape (41007, 470) (2563, 59) Dask graph 128 chunks in 2 graph layers Data type

  • value
    (time, basin_id)
    float32
    dask.array<chunksize=(5126, 59), meta=np.ndarray>

    Array Chunk Bytes 73.52 MiB 1.15 MiB Shape (41007, 470) (5126, 59) Dask graph 64 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (2)
  • basin_id
    PandasIndex

    PandasIndex(Index(['010101', '010801', '010802', '010901', '010902', '010903', '011001',
           '011002', '011003', '011201',
           ...
           '103704', '103714', '103715', '103801', '104401', '104801', '104804',
           '120201', '132901', '135201'],
          dtype='object', name='basin_id', length=470))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1910-01-01', '1910-01-02', '1910-01-03', '1910-01-04',
                   '1910-01-05', '1910-01-06', '1910-01-07', '1910-01-08',
                   '1910-01-09', '1910-01-10',
                   ...
                   '2022-03-31', '2022-04-01', '2022-04-02', '2022-04-03',
                   '2022-04-04', '2022-04-05', '2022-04-06', '2022-04-07',
                   '2022-04-08', '2022-04-09'],
                  dtype='datetime64[ns]', name='time', length=41007, freq=None))

• Attributes: (0)